Sox, Harold C. MD, MACP
From the Dartmouth Medical School, NH.
Reprints: Harold C. Sox, MD, MACP, 31 Faraway Lane, West Lebanon, NH 03784. E-mail: email@example.com.
Defining comparative effectiveness research (CER) was the first order of business for the Institute of Medicine Committee on Initial Priorities for CER. The Institute of Medicine committee approached the task of defining CER by identifying the common theme in the 6 extant definitions.
The definition follows:
“Comparative effectiveness research is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”
The key words in this definition are “generation and synthesis of evidence” (which implies both original research and systematic reviews), “alternative methods” (which implies making head to head comparisons in study populations typical of daily practice), and “to make informed decisions” (which implies a focus on data that helps to decide between alternatives).
Defining CER requires us to decide what we want from decisions about health care. Definitions also serve a bureaucratic function: they can set boundaries that delineate which research is eligible for CER program funding. Definitions—and the funding that advances their goals—can reshape the research environment.
This brief essay begins a Medical Care Supplement about comparative effectiveness research (CER). The Supplement will highlight some of the methodological challenges of CER. The purpose of this introductory essay is to define CER. I hope that the reader will think about whether the research described in the Supplement satisfies the definition of CER.
Definitions are important for several reasons. They can serve to focus our attention on our goals, which in this case are the goals of a nation in which runaway health care costs are a threat to recovery from a very severe economic recession. Defining CER was the first order of business for the Institute of Medicine (IOM) Committee on Initial Priorities for CER. We should ask whether the IOM definition really serves the public interest or whether we need to change it while we still can, before it becomes ingrained in the conduct of our affairs. The definition of CER also has a bureaucratic function. It sets administrative boundaries: it can define which research proposals are eligible for CER research funding, shape the research proposals, and provide the framework by which a study section decides which research proposals best advance the goals implied by the definition. Definitions—and the funding that advances their goals—can reshape the research environment.
The American Recovery and Reinvestment Act of 2009 charged the IOM committee to solicit nominations of research questions and set priorities among the nominations. The ultimate goal was to develop a research agenda that responded to the needs of the American public and could set the course for CER for a decade or longer while directing perhaps billions of dollars of research support. The committee wrote a definition of CER because it needed a criterion for including nominated topics in its priority setting process or excluding them.
THE IOM DEFINITION OF CER
The IOM committee approached the task of defining CER by identifying the common theme in the 6 extant definitions1 and trying to express them concisely. The definition follows:
“Comparative effectiveness research is the generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor the or improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.”
These words taken together define a form of research that would constitute a unique mission for a well-funded national program. Calls for a national program of CER come from many directions, including health economists2 and a previous IOM committee.3 At this writing (October 2009), both the U.S. Senate and House of Representatives versions of health reform legislation contain provisions for a national CER program.4,5 Therefore, we should pay attention to a definition around which a well-funded national program would organize itself.
Some of the words are especially important.
GENERATION AND SYNTHESIS OF EVIDENCE
“Generation” means so-called “primary research:” hypotheses, newly collected and analyzed data, and conclusions. A “synthesis of evidence” (or systematic review) starts with an exhaustive search for all completed research, systematically summarizes it, and characterizes its strengths and shortcomings. According to the draft legislation now before the U.S. Congress, the proposed national CER program would support both of these forms of research.4,5 Over the past several decades, clinical policymakers (those who develop practice guidelines, performance measures, and insurance coverage policy) have increasingly based their recommendations on systematic reviews rather than less rigorous forms of evidence summary.
EFFECTIVENESS OF ALTERNATIVE METHODS
These 4 words encompass 2 ideas. First, “effectiveness”—as contrasted with “efficacy:”—implies studies done in patients who are typical of those seen in day-to-day practice. Effectiveness studies have few exclusion criteria. This constraint will maximize the external validity (generalizability and applicability) of the study. It will address more effectively the practicing physician's question “do the results apply to patients in my practice.”
Studying patients typical of daily practice will mean making inferences from studies of patients cared for in daily practice, so-called observational research. Making reliable inferences about effectiveness from observational research will require advances in statistical methods and more systematic capture of clinical data. Pragmatic (or practical) clinical trials are another part of the solution. Their explicit purpose is to inform decision makers by studying typical interventions and clinically important outcomes in typical populations.6
The second idea is the phrase “alternative methods,” which implies that the research will compare at least 2 interventions that are effective enough to be the standard of practice. The words imply “head-to-head” comparisons as distinguished from comparing an active intervention with a placebo. Head-to-head comparisons answer the question that many practitioners ask: “which of these 2 treatments is best?”
MAKING INFORMED HEALTH DECISIONS
This phrase is the strongest rationale for the claim that CER is something new. It implies a focus on how physicians and patients make decisions, what evidence will best inform decision making, and how increase the chance that physicians and patients will use the evidence effectively. Clinical studies must seek the information that will help to identify which patients will benefit most from test or treatment A and which will benefit most from test or treatment B. Getting this information is feasible, and ample precedent exists.7
In addition, we must learn to translate the term “patient centeredness” into actions that promote good decisions. Patient centeredness will require us to know patients' preferences for treatments and the health states they may experience. We can then use the principles of expected utility decision making to forecast the consequences of the tests or treatments from which the patient must choose. As the public starts to expect patient-centered care every time, decision quality will become a key measure of health system performance. Decision quality measures are in their infancy.
Defining CER forces decision makers—health professionals and patients—to identify the information that they need. The definitions of CER all focus on making head-to-head comparisons in study populations that are typical of clinical practice. That health professionals seem to agree on these attributes of the inputs to decision making is reason to celebrate. The IOM definition of CER also emphasizes decision making per se, which implies attention to the process of decision making.
1. Institute of Medicine (IOM). Initial National Priorities for Comparative Effectiveness Research
. Washington, DC: The National Academies Press; 2009.
2. Institute of Medicine (IOM). Knowing What Works in Health Care: a Road Map for the Nation
. Washington, DC: The National Academies Press; 2008.
3. Wilensky GR. Developing a center for comparative effectiveness information. Health Aff (Millwood)
4. H.B. 3200. pp 501–524. Accessed October 29, 2009.
5. S. 1796, 001129–1192. Accessed October 29, 2009.
6. Luce BR, Kramer JM, Goodman SN, et al. Rethinking randomized trials for comparative effectiveness research: the need for transformational change. Ann Intern Med
7. Detre K, Peduzzi P, Murphy M, et al. Effect of bypass surgery on survival in patients in low- and high-risk subgroups delineated by the use of simple clinical variables. Circulation
© 2010 Lippincott Williams & Wilkins, Inc.