In December 2007, the Congressional Budget Office (CBO) released a report entitled “Research on the Comparative Effectiveness of Treatments.”1 The CBO noted that for many conditions high-quality evidence that can direct clinical care is lacking. The report called for expanded federal efforts in comparative effectiveness research (CER), defined as “a rigorous evaluation of the impact of different options that are available for treating a given medical condition for a particular set of patients.” Subsequent to the CBO report, 2 major pieces of congressional legislation have stimulated public interest in CER. The American Recovery and Reinvestment Act of 2009 allocated $1.1 billion to the National Institutes of Health (NIH), the Agency for Health Research and Quality (AHRQ), and the Office of the Secretary for Health and Human Services to fund comparative effectiveness research and to generate major reports on national priorities in CER. The Affordable Care Act of 2010 established a nonprofit nongovernmental “Patient Centered Outcomes Research Institute”2 that will establish and oversee a national research program and attempt to establish methodological standards for CER.
While the CBO report and the 2 congressional acts have drawn much attention to CER, such research is not new.3 The NIH and the Veterans Administration (VA) have supported comparative effectiveness research for decades. However, the new national focus on CER has stimulated a number of discussions—and, at times, debates4—on topics such as the need for more high-quality clinical evidence, the attitudes of physicians towards science and evidence-based decision-making,5 the role of economics research in health care,6 and (among the most contentious issues) the complementary roles of observational and experimental research in informing clinical care.7,8 With the advent of the informatics revolution and advances in biostatistic methods, some are asking whether population-based and clinical epidemiology may be in a position to push aside the “gold standard” status traditionally given to randomized trials. Others are more cautious, noting that observational findings have often led to dangerously wrong conclusions.
Fifty years ago, NIH-funded Framingham Heart Study investigators published a classic paper in which they described how electrocardiographic left ventricular hypertrophy, hypercholesterolemia, and hypertension predicted risk of symptomatic coronary heart disease.9 Commenting on the clinical utility of their findings, the investigators wrote, “There can be no doubt that absence of these characteristics is distinctly advantageous since such persons demonstrate a relatively low risk of developing CHD.”9 p. 48 But they went on to caution, “Whether or not the correction of these abnormalities once they are discovered will favorably alter the risk of development of disease, while reasonable to contemplate and perhaps attempt, remains to be demonstrated [italics added].”9 p. 48
Over the next few decades, a number of randomized trials, some funded by NIH, went on to test whether correction of hypercholesterolemia and hypertension favorably altered risk. The Lipid Research Clinics Coronary Primary Prevention Trial10 and the Program on the Surgical Control of Hyperlipidemias11 demonstrated that cholesterol reduction prevents fatal and nonfatal coronary heart disease events. Similarly, the Systolic Hypertension in the Elderly Project12 demonstrated that pharmacologic reduction of systolic blood pressure reduces the risk of stroke, other major cardiovascular events, and all-cause death. Later, the Antihypertensive and Lipid Lowering to Prevent Heart Attack (ALLHAT) trial showed that initial therapy with thiazide diuretics was as effective as newer and more expensive antihypertensive agents in its effect on clinical outcomes.13 All of these studies were funded by NIH.
Today, NIH continues to support research that aims to inform clinicians and other decision-makers who seek optimal strategies to manage hypercholesterolemia and hypertension. The recently completed ACCORD trial assessed the value of fibrates and aggressive blood pressure lowering in patients with complicated Type 2 diabetes mellitus.14 The ongoing Systolic Pressure Intervention Trial (SPRINT) will determine the value of aggressive blood-pressure lowering in nondiabetic patients with systolic hypertension. Meanwhile, the NIH is also funding an 800,000-patient clinical epidemiology investigation of the management of hypertension within a network of health maintenance organizations.15
In 1984, the NIH-funded Multi-Center Postinfarction Research Group published a report describing how left ventricular systolic dysfunction and frequency of ventricular arrhythmias predicted 2-year death rates among patients who survived myocardial infarction.16 In their discussion, the authors cautioned, “The present study indicates that ventricular arrhythmias do pose an independent risk in the first 3 years after myocardial infarction, but we cannot make any statement about the likelihood of improvement in mortality figures conditional on antiarrhythmic drug treatment [italics added]. Our results do encourage the pursuit of further studies to determine the benefit/risk ratio for treatment of frequent or repetitive VPDs [ventricular premature depolarizations].”16 p. 256
Five years later, investigators from the NIH-funded Cardiac Arrhythmia Suppression Trial published their main findings: to the surprise of many, antiarrhythmic drugs not only failed to reduce death rates, but actually increased the risk of death among survivors of myocardial infarction.17 Over time, clinical scientists came to accept that antiarrhythmic drug therapy yielded little if any benefit. Instead, as later demonstrated by the Sudden Cardiac Death in Heart Failure Trial,18 mortality can be reduced with implantable defibrillators. Today, the NIH is funding a large-scale clinical epidemiology project that seeks to identify characteristics of patients who qualify by current guidelines for an implantable defibrillator, but may live just as long without one. Like the hypertension project, this study is based in a network of health maintenance organizations.
WHAT THESE STORIES TEACH US
The stories of hypertension, hypercholesterolemia, and ventricular arrhythmias highlight 6 key points regarding the proper roles of population-based and clinical epidemiology in comparative effectiveness research.
First, epidemiologic studies are valuable for generating hypotheses. Epidemiologic discoveries have led to hypotheses supporting reduction of high blood pressure, high cholesterol, and frequent ventricular arrhythmias.
Second, prudent epidemiologists appropriately caution that their observationally derived hypotheses should not be extrapolated directly into policies for clinical practice. Sometimes effect sizes are enormous, as in the cases of smoking and lung cancer, and diethylstilbestrol and clear cell carcinoma, obviating a need for randomized trials. Much more often, though, the associations between exposures and outcomes are modest, necessitating rigorous large-scale randomized trials to account for unmeasured confounders. Randomized trials showed that blood-pressure- and cholesterol-lowering drugs did in fact improve clinical outcomes, but antiarrhythmic agents did not. We have seen cases where observational findings were misleading3; postmenopausal hormone therapy does not prevent coronary heart disease events, routine stenting after completed myocardial infarction does not improve long-term outcomes, and high-dose chemotherapy for metastatic breast cancer does not improve survival.
Third, clinical epidemiology studies are helpful to extend the results of clinical trials, including studying how clinical interventions that are effective in RCTs can be implemented and disseminated in routine practice, and exploring possible heterogeneity in treatment effects and harms in subpopulations. Epidemiologic investigations can also be used to compare strategies that may be difficult to study in randomized trials. For example, investigators have used instrumental variables to assess the importance of timing of invasive therapies in acute coronary syndromes,19 large-scale registries to compare practices and outcomes of procedures as a function of physician specialty,20 and quasi-experimental approaches to assess the impact of passive-smoking laws on public health.21
Fourth, the public sector plays a critically important role in generating and supporting comparative effectiveness research.3,4 Many of the hypotheses for CER are based on epidemiologic studies funded by governmental agencies. Major trials as well as epidemiologic research are funded by the government when there is no compelling financial interest that attracts private sector support. Presently, the NIH continues to play a leading role in both observational and experimental CER,4 supporting major clinical epidemiology efforts such as the Nurses' Health Study and the HMO Research Network; population-based studies such as the Atherosclerosis Risk in Communities (ARIC) study and the National Cancer Institute's Surveillance, Epidemiology and End Results (SEER) Program; large-scale randomized trials such as the National Lung Screening Trial; and projects designed to enhance innovative CER methods such as instrumental variable analysis and Bayesian trials. The NIH recently allocated $22 million for research on CER methods—methods that may benefit the conduct of classic epidemiologic investigations.
Fifth, comparative effectiveness research and epidemiology can operate in concert to improve public health. Cardiovascular mortality has declined by over 70% since the first Framingham findings were published. A number of reports suggest that one-third of the decline can be attributed to high technology medical care (eg, defibrillators), one-third to medical management of hypertension and cholesterol, and one-third to behavioral changes such as marked reductions in cigarette smoking. Discoveries made by epidemiologists and trialists played critical roles in all 3 of these domains.22
Finally, while this essay has largely focused on drugs, epidemiology and CER can inform research directions and practice for varied kinds of interventions. Clinical epidemiology and randomized trials have produced insights and evidence about screening for surgically treated conditions such as abdominal aortic aneurysms,23 about behavioral interventions to prevent progression of glucose intolerance to frank diabetes mellitus,24 and about changes in health care delivery to prevent complications of maternal labor.25
Despite the controversies, a look at the history of comparative effectiveness research demonstrates that we have much to be proud of. As the cardiovascular risk story and many other stories attest, government agencies, epidemiologists, and trialists working together can revolutionize biomedical science, turning it into an incredibly powerful force for better health.
ABOUT THE AUTHORS
MICHAEL LAUER is Director of the Division of Cardiovascular Sciences at the National Heart, Lung, and Blood Institute; previously he was Professor of Medicine, Epidemiology, and Biostatistics at the Cleveland Clinic Lerner College of Medicine of Case Western Reserve University. RICHARD HODES is the Director of the National Institute of Aging. Richard Hodes and Michael Lauer are co-chairs of the trans-NIH Comparative Effectiveness Research Coordinating Committee.
2.Clancy C, Collins FS. Patient-Centered Outcomes Research Institute: the intersection of science and health care. Sci Transl Med
3.Lauer MS. The historical and moral imperatives of comparative effectiveness research. Stat Med
. 2010;29:1982–1984; discussion 1996–1997.
4.Lauer MS, Collins FS. Using science to improve the nation's health system: NIH's commitment to comparative effectiveness research. JAMA
6.Garber AM. A menu without prices. Ann Intern Med
7.Tunis SR, Benner J, McClellan M. Comparative effectiveness research: policy context, methods development and research infrastructure. Stat Med
8.Pocock SJ, Elbourne DR. Randomized trials or observational tribulations? N Engl J Med
9.Kannel WB, Dawber TR, Kagan A, Revotskie N, Stokes J III. Factors of risk in the development of coronary heart disease—six year follow-up experience. The Framingham Study. Ann Intern Med
10.The Lipid Research Clinics Coronary Primary Prevention Trial results. I: reduction in incidence of coronary heart disease. JAMA
11.Buchwald H, Varco RL, Matts JP, et al. Effect of partial ileal bypass surgery on mortality and morbidity from coronary heart disease in patients with hypercholesterolemia. Report of the Program on the Surgical Control of the Hyperlipidemias (POSCH). N Engl J Med
12.SHEP Cooperative Research Group. Prevention of stroke by antihypertensive drug treatment in older persons with isolated systolic hypertension. Final results of the Systolic Hypertension in the Elderly Program (SHEP). JAMA
13.Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic: the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA
14.Ginsberg HN, Elam MB, Lovato LC, et al. Effects of combination lipid therapy in type 2 diabetes mellitus. N Engl J Med
15.Go AS, Magid DJ, Wells B, et al. The Cardiovascular Research Network: a new paradigm for cardiovascular quality and outcomes research. Circ Cardiovasc Qual Outcomes
16.Bigger JT Jr, Fleiss JL, Kleiger R, Miller JP, Rolnitzky LM. The relationships among ventricular arrhythmias, left ventricular dysfunction, and mortality in the 2 years after myocardial infarction. Circulation
17.Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. The Cardiac Arrhythmia Suppression Trial (CAST) Investigators. N Engl J Med
18.Bardy GH, Lee KL, Mark DB, et al. Amiodarone or an implantable cardioverter-defibrillator for congestive heart failure. N Engl J Med
19.Ryan JW, Peterson ED, Chen AY, et al. Optimal timing of intervention in non-ST-segment elevation acute coronary syndromes: insights from the CRUSADE (Can Rapid risk stratification of Unstable angina patients Suppress ADverse outcomes with Early implementation of the ACC/AHA guidelines) Registry. Circulation
20.Curtis JP, Luebbert JJ, Wang Y, et al. Association of physician certification and outcomes among patients receiving an implantable cardioverter-defibrillator. JAMA
21.Pell JP, Haw S, Cobbe S, et al. Smoke-free legislation and hospitalizations for acute coronary syndrome. N Engl J Med
22.Cutler D, Kadiyala S. The return to biomedical research: treatment and behavioral effects. In: Murphy KM, Topel RH, eds. Measuring the Gains From Medical Research: An Economic Approach
. Chicago: University of Chicago Press; 2003:110–162.
23.Ashton HA, Buxton MJ, Day NE, et al. The Multicentre Aneurysm Screening Study (MASS) into the effect of abdominal aortic aneurysm screening on mortality in men: a randomised controlled trial. Lancet
24.Tuomilehto J, Lindstrom J, Eriksson JG, et al. Prevention of type 2 diabetes mellitus by changes in lifestyle among subjects with impaired glucose tolerance. N Engl J Med
25.Althabe F, Buekens P, Bergel E, et al. A behavioral intervention to improve obstetrical care. N Engl J Med