In 2006, the Clinical and Translational Science Award (CTSA) program was initiated by the National Institutes of Health (NIH) with the goal of increasing and enhancing clinical and translational research at U.S. academic institutions, and accelerating and maximizing the impact of scientific discovery to improve the public's health.1 This transformative program seeks to achieve these goals through education and training, research support and infrastructure (including informatics), community engagement, methods development, and adoption of a cross-cutting approach to research that spans diseases, departments, and disciplines. Since the program began, 55 of a planned 60 institutions have been funded by the CTSA mechanism, and a robust national consortium has grown. The CTSA program encourages academic institutions to recognize and value applied scholarship that seeks to improve health outcomes, including what has become known as comparative effectiveness research (CER) or patient-centered outcomes research.
The Patient Protection and Affordable Care Act of 2010 (PPACA) increased recognition of the importance of evidence-based practice informed by CER. In the period leading up to the passage of the PPACA, this previously underappreciated2 component of the translational research enterprise received tremendous attention within national planning efforts, and several panels proposed definitions and agendas for CER.3,4 Following those efforts, a leadership committee of CTSA investigators developed a white paper on how the CTSA consortium can facilitate CER.5 They recognized that CTSA sites are poised to support CER efforts through their relevant clinical trials infrastructure, education and training programs, community engagement efforts, methods development, and informatics support. CTSA sites also have the advantage of existing and growing cross-institutional and cross-regional linkages and infrastructure.
As this national committee developed its white paper, the leadership of the Albert Einstein College of Medicine of Yeshiva University (Einstein) engaged faculty in a comprehensive review and update of its institutional strategic research plan in 2010. As part of this endeavor, our CTSA— a formal partnership with the Montefiore Medical Center (Montefiore) and known locally as the Einstein- Montefiore Institute for Clinical and Translational Research (ICTR)—was tasked to conduct a focused review and develop a set of recommendations to enhance our CER portfolio. In this article, we begin with a brief overview of the U.S. government's efforts in CER, and then describe the process we undertook at our institution to enhance CER within the context of this history, our institution's resources and strategic plan, and the CTSA program. This article provides our CTSA site's institutional response to this challenge as an example with potential broader applicability.
A Brief Overview of U.S. Government Efforts in CER
CER, evidence-based medicine, and technology assessment share an intertwined, interrupted, renamed, and politicized history, beginning with James Lind's first controlled clinical trial of six different treatments for scurvy in 1747. Even today's Joint Commission grew from Ernest Codman's systematic pursuit of long-term “end results” for surgical patients, “outcomes management,” and standardized hospital care early in the 20th century; its annual achievement awards for improving the quality and safety of patient care were named for him in 1996.
With the passage of Medicare and Medicaid in 1965, the U.S. government became a major actor in health care. In 1968, the U.S. Department of Health and Human Services established the National Center for Health Services Research and Development, which later became the Bureau of Health Services Research (1973) and then the National Center for Health Services Research (NCHSR; 1975–1985). NCHSR added Health Care Technology Assessment (HCTA) in 1985.6 In 1989, the NCHSR-HCTA became the Agency for Healthcare Policy and Research, which supported research teams that produced and published some 19 guidelines from 1992 to 1996. However, when the guidelines on low-back pain found insufficient evidence to support certain spinal surgeries, strong opposition from surgeons led to pressure to eliminate the agency. The agency instead suffered dramatic budget cuts in 1996, and it was renamed the Agency for Healthcare Research and Quality (AHRQ).7 In 1998, with the American Medical Association and the American Association of Health Plans, the AHRQ launched the National Guideline Clearinghouse (www.guideline.gov), an online database of evidence-based clinical practice guidelines that continues today and offers more than 2,400 guideline summaries.
In 1972, the Office of Technology Assessment was created as an advisory arm of the U.S. Congress. The Office of Technology Assessment “had a presence and influence in many of the great scientific debates on Capitol Hill” in which its studies “dealing with issues like medical research... played pivotal roles.”8 The agency closed in 1995, after losing its funding, which some attributed to “shortsightedness about its value in providing unbiased, understandable advice on complex issues.”8
From 1978 to 1981, a parallel National Center for Healthcare Technology in the Department of Health, Education, and Welfare/Health and Human Services sponsored major evaluations of coronary artery bypass surgery, cesarean sections, and dental radiology and made some 75 recommendations to the Medicare program about coverage. The center ended with the change of administrations.
In 2003, the Medicare Prescription Drug, Improvement, and Modernization Act authorized the AHRQ to conduct research with a focus on “outcomes, comparative clinical effectiveness, and appropriateness of health care items and services” for Medicare and Medicaid enrollees.9 With funds appropriated in 2005 under Section 1013 of the act, the AHRQ created the Effective Health Care program that reviews and synthesizes existing evidence through evidence-based practice centers, generates new information using approved research centers and networks, and publishes findings in formats addressing the differing needs of policy makers, clinicians, and patients of the Medicare, Medicaid, and State Child Health Insurance Program.10
On February 17, 2009, the American Recovery and Reinvestment Act (ARRA) became law and appropriated $1.1 billion for CER, divided among the NIH ($400 million), the AHRQ ($300 million), and the Secretary of Health and Human Services ($400 million, administered by AHRQ).11 The legislation called on the Institute of Medicine (IOM) to recommend research priorities for these funds and gather stakeholder input. The IOM's report, which listed 100 initial topics for study, was delivered June 30, 2009.3 In addition, ARRA created the Federal Coordinating Council for Comparative Effectiveness Research (FCCCER) to offer guidance and coordination on the use of these funds.11 The FCCCER's report to the president and Congress, also delivered June 30, 2009, noted that NIH historically has been the largest source of federal support for CER; its preliminary inventory found 463 CER studies funded by the NIH, 144 by the AHRQ, 96 by the Veterans Administration, and 25 by the Department of Defense during fiscal years 2006–2009.4
On March 23, 2010, President Barack Obama signed the PPACA into law, which added $3 billion in CER funding and created a new Patient-Centered Outcomes Research Institute (PCORI), with its own trust fund, that will specialize in comparative studies of medical interventions.12 The legislation establishes a board of governors (which includes the directors of the AHRQ and the NIH as well as nongovernment members who represent patients, health care providers, insurers, industry, etc.), expert advisory panels for clinical trials and rare diseases, and a methodology committee. The law explicitly limits cost-benefit and cost-effectiveness research, saying that the PCORI “shall ensure that [its] research findings... not be construed as mandates for practice guidelines, coverage recommendations, payment, or policy recommendations.”12
The November 2010 midterm elections resulted in a change in the majority in the House of Representatives from the Democrats to the Republicans, who ran on a slogan pledging to “Repeal and Replace” the health care reform law. (At the time of this writing, the House had passed repeal legislation.) The PPACA is also facing multiple legal challenges.13,14 Thus, federal funding for CER remains in flux. Meanwhile, despite politicization, the NIH is working to maintain the scientific integrity of CER.15
It seems promising that recently there has been much attention paid to CER as a result of the anticipated changes to the U.S. health care system and that efforts have been made to enhance the funding for this research. However, the history described above should serve as a cautionary tale. Academic medicine needs to be responsive to changes in an evolving system without building its house on shifting sands. Indeed, plans to move the CTSA program from the National Center for Research Resources to a new National Center for Advancing Translational Science at the NIH16 show that institutions responding to changes in the health care and research funding systems need to be adaptable and forward-looking.
A Vision for CER at the Einstein-Montefiore ICTR
Context for our institutional efforts
In conceptualizing our response to the challenges of conducting and funding CER, the leaders of our CTSA held a mini-retreat in January 2010 to develop our broad-strokes vision for CER at the Einstein-Montefiore ICTR. We engaged in a far-reaching discussion, seeking to identify our institutional strengths, weaknesses, opportunities, and challenges. This discussion was framed by the FCCCER definition of CER as “the conduct and synthesis of research comparing the benefits and harms of different interventions and strategies to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings.”4
We recognized that the FCCCER definition of CER looks beyond the methodologies and investigations traditionally subsumed under the rubric of “health services research,” which is defined as follows by the AHRQ: “Health services research examines how people get access to health care, how much care costs, and what happens to patients as a result of this care.”17
Developing a model
In preparation for an intensive update and revision of Einstein's strategic research plan, a broadly representative ICTR task force on CER was established in February 2010. The task force included administrative leaders from both Einstein and Montefiore as well as a broad range of investigators with expertise in clinical trials, epidemiology, behavioral sciences, T1 translational research, social sciences, community-based participatory research, and evaluation and implementation sciences. This task force generated a report that developed an innovative integrated and multidisciplinary model for considering CER, which goes beyond the traditional thinking that equates CER with health services research (Figure 1).
Our model underscores the importance of multiple disciplines and methodologies: In addition to those encompassed by health services research, we emphasize the critical importance of efficacy studies and clinical trials, biobehavioral research and prevention, and social science and implementation research. Our approach to CER seeks to go beyond examination of clinical impact on individuals to include a multilevel, systems view of studying achievement of health improvement at both the individual and population levels. This means unpacking the black box of interventions and expanding essential CER questions to assess not only what works best when and for whom but also to explore how these interventions work in individuals, families, and communities. Our model gives explicit attention to issues of context, dissemination, translation, adoption, and maintenance, and it includes a strong emphasis on behavioral and social factors.
Connections Among CTSA Programs and Other Institutional Resources
Armed with this conceptual model, our task force sought to identify areas where ICTR programs and resources could advance our institutional CER efforts. Like all CTSA sites, the ICTR includes an education, training, and career development program; a biostatistics core; an informatics core; clinical research units; biobanks and biorepositories; a biomarker analytic resource; a novel methodologies core; a community engagement core; support for pilot research projects; and an administrative support unit. ICTR programs, like programs at many other CTSA sites, connect strongly to efforts throughout our academic health center. The programs and resources described below represent the linkages, keyed to Figure 2, among the ICTR and other programs that connect across organizational boundaries to advance the mission of CER.
CTSA education and training programs.
Our CTSA's educational offerings include courses, a certificate program, and degree programs, and career development funding is available. In 2010, we received an ARRA-funded supplement grant to develop training initiatives to enhance the research workforce for CER.
Center for Public Health Sciences.
This entity initiated a novel master's of public health program in 2010 that emphasizes community-based research conducted in multidisciplinary teams, with a focus on the behavioral and social determinants of health. The MPH program is directed by a sociomedical scientist whose research follows the community-based participatory research model.18 She is a coinvestigator on the ARRA-funded CER workforce supplement grant and has actively embraced CER as strongly connected to her vision for MPH training.
CTSA community engagement program.
This program, which is closely linked to the public health and community health programs, supports CER research efforts: For example, a recent NIH grant to Einstein and Urban Health Plan (also in the Bronx) funds a collaborative effort to create an infrastructure to support CER activities in a network of federally qualified health centers. Our affiliated practice-based research network—the New York City Research and Improvement Networking Group, with 20 primary care practices in the Bronx, Manhattan, and Yonkers—has strong interests in CER and is conducting two NIH-funded randomized clinical trials comparing interventions to promote breastfeeding.
Office of Community Health.
The creation of an Office of Community Health at Montefiore in July 2009 will facilitate CER. This entity was one of the direct outcomes of Montefiore's recent strategic planning process, which was undertaken in 2008 with the active involvement of Einstein and CTSA leadership. Its mission is “to improve the health of the communities we serve,” and its charge is to help coordinate efforts to intervene on a clinical and population level to describe, evaluate, and seek ways to improve health outcomes for one of the poorest urban areas in the United States, at the interface between clinical care and community/population health.19,20 In addition, Montefiore recently embarked on a high-level strategic initiative to become an “accountable care organization”21 and to develop replicable models of the patient-centered medical home22 that can be fully described and evaluated. The development of these clinical care delivery models will be informed by the active involvement of ICTR research staff and resources, in areas including methodology and study design, outcomes and HSR, and implementation and intervention research.
Einstein strategic research plan.
Einstein developed a strategic research plan in 2007,23 which was reviewed and updated in 2010. Participating faculty were divided into six major “research theme” groups, among which was CER. The CER working group, armed with the model of CER we developed (Figure 1), proposed the creation of a new center to support CER and provided a detailed list of faculty recruitment, research core, and staffing needs. Central to this strategic initiative is a coordinated effort across the academic health center, in which the medical school works with the medical center to forge a true clinical research enterprise in conjunction with the patient care mission. During a daylong retreat in April 2010, the faculty embraced this vision, with this center included among the top priorities in the strategic plan update.24
Blood and tissue banking has become a high priority as clinical and translational research has moved to utilize advances in clinical genomics, proteomics, and biomarkers. To help support the research community's requirement for archival storage of samples on well-characterized human subjects, the ICTR supports a biospecimen repository that functions in a federated model: Subrepositories are dedicated to specific research entities, but access is provided to secondary users. The biospecimen repository operates in conjunction with the informatics core to ensure secure specimen linking to clinical databases derived from the electronic medical record (Montefiore's clinical information system) or other phenotyping of human subjects and patients. The repository provides an essential resource for performing CER with a focus on the importance of biomarkers, genetics, and molecular diagnostics.
Synergy with disease-focused research centers.
Like all CTSA sites, our ICTR is agnostic regarding diseases or organ systems, but we recognize the value of collaborating with disease-focused research centers. For example, Einstein's NIH-funded Diabetes Research Center includes a prevention and control core (P&C) that provides the necessary linkages from translational research, through efficacy studies and epidemiology, into the realm of applied effectiveness and translation for broad implementation. Many of the studies supported by our P&C over the past 15 years are truly CER in nature—they are often patient-centered and include cost evaluations of various behavioral or educational interventions for potential dissemination.25,26
Clinical information system.
Our CIS—which has been implemented across a wide-ranging network of hospital and community-based sites in the Bronx and has historically included approximately 2.3 million unique patient records, used for analytic and operational purposes—offers unprecedented opportunities for CER through clinical and population health monitoring and intervention. The Montefiore Health System encompasses acute, postacute, ambulatory, home health, and community care delivery settings, advanced information technology systems, and varied payment models, including capitation. This integrated health care delivery system therefore offers an ideal “implementation laboratory” to conduct operational and evaluation activities that use this resource.
As an example, Montefiore secured competitive state funding in 2007 to implement a pay-for-performance demonstration program across its acute care and ambulatory sites. The program uses financial incentives to reward performance on measures of care for prevalent community conditions including diabetes mellitus, cardiovascular disease, and cardiovascular disease risk factors. The potential of such pay-for-performance programs to create disincentives to care for vulnerable populations and potentially worsen health care disparities has been noted.27 Montefiore is using its recently redesigned demographic data collection processes to evaluate quality of care provided to prevalent Bronx demographic subgroups, such as African Americans and Hispanics/Latinos. In addition, many other CER studies have used data from Montefiore's CIS.28–31
Programs in development.
Several new initiatives, strongly connected to our CTSA's development plans for CER, include the establishment of a Center for Health and Society (called for in Einstein's revised strategic research plan, as described above), and an Office of Clinical Trials.
Implementing and Evaluating the CER Model
We developed the conceptual model described above over a three-month period (November 2009 to January 2010), fleshed out the implementation strategy over the next three months (February to April 2010) in concert with Einstein's strategic research plan revision process, and have continued capacity-building and development efforts since then (e.g., obtaining ARRA funding for a workforce development supplement grant). What we have described is clearly an early-stage work in progress for which outcomes metrics are not yet available. The CER program will be implemented over several years and will require us to build additional infrastructure, acquire funding, and recruit faculty (Figure 3).
To monitor the development and evolution of CER capacity within the ICTR and its academic community, we have developed a multistepped evaluation plan that will
- create logic models for each component to prioritize short-, intermediate-, and long-term measures that can be used to track progress,
- conduct routine semiannual, semistructured interviews with core directors to document how their expertise is being utilized by Einstein faculty and other cores,
- conduct a network analysis through ongoing data collection to ascertain which cores are being approached for assistance and by whom, and
- conduct, via a faculty survey and use of institutional and national databases, an annual review of CER grants applied for, CER presentations, and CER publications by Einstein faculty.
Consistent with the goals of CER, we will seek to go beyond these academic metrics to evaluate the real-world impact of our CER findings on practice, policy, and outcomes. Using this framework, as we gain experience, we will also gain a better understanding of our model's “comparative effectiveness” for furthering CER and clinical and translational research.
Through the CTSA planning efforts detailed in this article, over the past 18 months we have gone through the initial process of strategic alignment (3 months) and framework development and elucidation of a model for CER (3–6 months), and we have begun the capacity-building process of grant applications, educational program development, faculty recruitment, and infrastructure development (9 months). Our work has occurred in parallel with a national effort to reinvigorate CER through health care reform and the evolving CTSA program. Although the landscape of health care and research funding is changing, we have found that the core mission of CER—identifying ways “to prevent, diagnose, treat and monitor health conditions in ‘real world’ settings”4—aligns with our institution's mission, values, and resources, allowing us to set a robust and useful strategic plan to fulfill our vision. Although we are still in the early days of this effort, we believe our process demonstrates the value of a medical school and an academic medical center collaborating through a CTSA to respond to the new opportunities provided under health reform. Thus, we share this model, in development, with other academic medical centers, medical schools, and CTSA sites, in the hope that we can collectively develop a scientifically grounded approach to improving health and health care.
The authors gratefully acknowledge the contributions of Stacey Sodlosky, MS, and Eileen M. Shore, MS, to this CER program, and of Alexandra Covino in preparing the manuscript.
This manuscript was partially supported by a CTSA grant from the National Center for Research Resources of the NIH, grant #UL1RR025750.
1Zerhouni EA. Translational and clinical science: Time for a new vision. N Engl J Med. 2005;353:1621–1623.
2Woolf SH. The meaning of translational research and why it matters. JAMA. 2008;299:211–213.
4U.S. Department of Health and Human Services. Federal Coordinating Council for Comparative Effectiveness Research: Report to the President and Congress. Washington, DC: U.S. Department of Health and Human Services; June 30, 2009. http://www.hhs.gov/recovery/programs/cer/cerannualrpt.pdf
. Accessed February 17, 2011.
5Selker HP, Strom BL, Ford DE, et al. White paper on CTSA consortium role in facilitating comparative effectiveness research. Clin Translat Sci. 2010;3:29–37.
6Machette RB, et al. Records of the Agency for Health Care Policy and Research. Record Group 510, 1964–87. Guide to Federal Records in the National Archives of the United States. Washington, DC: National Archives and Records Administration; 1995. http://www.archives.gov/research/guide-fed-records/groups/510.html
. Accessed November 29, 2010.
7Manchikanti L, Falco FJ, Boswell MV, Hirsch JA. Facts, fallacies, and politics of comparative effectiveness research: Part I. Basic considerations. Pain Physician. 2010;13:E23–E54.
8Leary WE. Congress's science agency prepares to close its doors. New York Times. September 24, 1995:26.
13Sack K, Pear R. Health law faces threat of undercut from courts. New York Times. November 27, 2010:A11.
14Sack K. Judge rejects health law challenge. New York Times. December 1, 2010:A24.
15Lauer MS, Collins FS. Using science to improve the nation's health system: NIH's commitment to comparative effectiveness research. JAMA. 2010;303:2182–2183.
16Kaiser J. A government niche for translational medicine and drug development. Science. 2010:330:1462–1463.
18Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity. Am J Public Health. 2010;100(suppl 1): S40–S46.
19Fielding JF, Teutsch SM. Integrating clinical care and community health. JAMA. 2009;302:317–319.
20Lurie N, Fremont A. Building bridged between medical care and public health. JAMA. 2009;302:84–86.
21Bohmer RM, Lee TH. The shifting mission of health care delivery organizations. N Engl J Med. 2009;361:551–553.
22Larson EB, Reid R. The patient-centered medical home movement: Why now? JAMA. 2010;303:1644–1645.
25Schechter CB, Basch CE, Caban A, Walker EA. Cost effectiveness of a telephone intervention to promote dilated fundus examination in adults with diabetes mellitus. Clin Ophthalmol. 2008;2:763–768.
26Jones HL, Walker EA, Schechter CB, Blanco E. Vision is precious: A successful behavioral intervention to increase the rate of screening for diabetic retinopathy for inner-city adults. Diabetes Educ. 2010;36:118–126.
27Casalino LP, Elster A, Eisenberg A, Lewis E, Montgomery J, Ramos D. Will pay-for-performance and quality reporting affect health care disparities? Health Aff (Millwood). 2007;26:w405–w414.
28Boucai L, Zonszein J. Effects of quality improvement strategies for type 2 diabetes in Bronx, N.Y. Clin Diabetes. 2007;25:155–159.
29Southern WN, Berger MA, Bellin EY, Hailpern SM, Arnsten JH. Hospitalist care and length of stay in patients requiring complex discharge planning and close clinical monitoring. Arch Intern Med. 2007;167:1869–1874.
30Abramowitz M, Muntner P, Coco M, et al. Serum alkaline phosphatase and phosphate and risk of mortality and hospitalization. Clin J Am Soc Nephrol. 2010;5:1064–1071.
31Southern WN, Drainoni ML, Smith BD, et al. Hepatitis C testing practices and prevalence in a high-risk urban ambulatory care setting [published online ahead of print May 20, 2010]. J Viral Hepat. doi:10.1111/j.1365-2893.2010.01327.x.