Share this article on:

Building Academic Health Centers' Capacity to Shape and Respond to Comparative Effectiveness Research Policy

VanLare, Jordan M.; Conway, Patrick H. MD, MSc; Rowe, John W. MD

doi: 10.1097/ACM.0b013e318217d0d1
Comparative Effectiveness Research

In recent years, the focus on comparative effectiveness research (CER), the funding available to support it, and the range of possible effects of CER policy on academic health centers (AHCs) have increased substantially. CER has implications for the research, education, and clinical care components of AHCs' missions. The current funding and policy environment have created specific opportunities for AHCs to shape and respond to CER policies across the four dimensions of the CER enterprise: research, human and scientific capital, data infrastructure, and translation and dissemination. Characteristics such as the degree of physician–hospital integration, the status of a health information technology infrastructure, and the presence of a well-developed cross-functional health services research capacity linked to the care delivery enterprise could help AHCs respond to these opportunities and influence future policies. AHCs are also essential to the development of methodologies and the training of the next cadre of researchers. Further, a focus on understanding what works in health care and increasing adoption of evidence-based practice must become embedded in the fabric of AHCs. Those AHCs most successful in responding to the CER challenge may leverage it as a point of differentiation in the marketplace for health care and lead transformational improvements in health.

At the time of writing, Mr. VanLare was program analyst, U.S. Department of Health and Human Services, Washington, DC. He is now a third-year medical student, Columbia University College of Physicians and Surgeons, New York, New York.

Dr. Conway is director of hospital medicine, associate professor of pediatrics, Division of General Pediatrics, assistant vice president for outcomes performance, and director of rapid evidence adoption, Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, Cincinnati, Ohio. He is former executive director, Federal Coordinating Council for Comparative Effectiveness Research, U.S. Department of Health and Human Services, Washington, DC.

Dr. Rowe is professor, Department of Health Policy and Management, Joseph L. Mailman School of Public Health of Columbia University, New York, New York. He is former president and CEO, Mount Sinai NYU Health, and former president, Mount Sinai School of Medicine of New York University and Mount Sinai Medical Center, New York, New York.

Correspondence should be addressed to Dr. Conway, 3333 Burnet Avenue, MLC 2011, Cincinnati, OH 45229; telephone: (513) 636-1945; e-mail:

First published online April 20, 2011

Since 2009, comparative effectiveness research (CER) has gained considerable attention in U.S. health care policy arenas as well among the general public. Press mentions of CER jumped from just over 200 in 2008 to more than 1,600 in 2010,1 and Google Internet searches surged over the same period.2 The enactment of the American Recovery and Reinvestment Act (ARRA) in 2009 injected an unprecedented $1.1 billion into CER.3 Adding to that one-time investment, the Patient Protection and Affordable Care Act (PPACA) of 2010 establishes a trust fund to support ongoing CER efforts between 2010 and 2019,4 and the proposed 2011 fiscal year budget for the Agency for Healthcare Research and Quality (AHRQ) contains $286 million for CER, a $261 million increase relative to pre-ARRA funding levels.5

In addition, significant progress has been made in establishing a consensus definition for CER (also known as patient-centered outcomes research) and an initial set of priorities for research and investment. The Federal Coordinating Council for Comparative Effectiveness Research (FCCCER), in its 2009 report to Congress, defined CER broadly (Box 1).6 The definition was developed using feedback that clinicians, academics, patients, and a wide variety of other stakeholders offered at three public listening sessions. The public also provided comments on a draft definition posted on the U.S. Department of Health and Human Services Web site during April to June 2009. The FCCCER's definition, finalized in June 2009, served as the basis for the definition of CER in the PPACA.7

Box 1

Box 1

The Institute of Medicine (IOM), in its 2009 report to Congress on CER, included a similarly broad definition of CER as well as a list of 100 priority research topics. The IOM produced the list using a robust, public process in which it solicited potential CER topics and subsequently ranked them on the basis of merit and need.8 While the mechanics by which these topics will be translated into funding, research, and improved care are still being developed, the list serves as a critical foundation for the CER agenda going forward.9 With respect to opportunities for academic health centers (AHCs)—institutions involved in clinical care, research, and health sciences education—it is important to note that a large proportion of the priorities identified by the IOM involve care delivery models. They focus less on studies of drug A versus drug B and more on how care is being delivered, by whom, and in what setting. Appropriate organizational structure and resources will enable AHCs to test interventions within the delivery system to improve patient outcomes.

Although the CER enterprise has gathered tremendous momentum in recent years, additional funding and priority setting are necessary, and stakeholders, including AHCs, must be engaged to improve patient care and the nation's health.10 The FCCCER6 identified four key dimensions of the CER enterprise—research, human and scientific capital, data infrastructure, and translation and dissemination (Figure 1)—and highlighted gaps in each that must be filled if CER is to improve the quality and efficiency of the health system. AHCs are unique in their ability to shape policy and close gaps across all four dimensions. Conversely, CER opportunities address all three pillars of the AHC mission to pursue novel research, educate health professionals, and provide clinical services to patients and the community. AHCs, therefore, are well positioned to take advantage of CER funding and research opportunities as well as to implement CER findings, thereby improving quality of care at their own institutions and in the communities they serve.

Figure 1

Figure 1

In this article, we discuss specific opportunities for AHCs to shape and respond to CER policy—both federal and that set by the new Patient-Centered Outcomes Research Institute—across the domains of research, human and scientific capital, data infrastructure, and translation and dissemination. Furthermore, we propose that AHCs develop a cross-functional role that integrates research and patient outcome improvement responsibilities to inform CER policy and adopt CER findings at the local level. The organizational structure that best addresses this role will typically be a cross-cutting center, or possibly a division, that focuses on CER issues and opportunities in the AHC's clinical, research, and educational components. This type of center would be the catalyst for change both in the local delivery system and through generalizable research. We use as examples established centers at Cincinnati Children's Hospital Medical Center (Cincinnati Children's) and Mount Sinai Medical Center (Mount Sinai).

Back to Top | Article Outline

AHCs and the Four Dimensions of CER


The Lewin Group counted more than 1,000 CER-related intramural and extramural supported studies from the U.S. government between September 2005 and June 2009. The majority of these studies (over 70%) were randomized controlled trials. Whereas many addressed medications (39%), fewer addressed behavioral interventions (21%), surgical procedures (8%), or devices (7%). Studies rarely addressed patients with multiple comorbidities, elderly adults, or other priority populations.11 A recent analysis of medication studies in six high-impact general medicine journals found that only 32% constituted CER and that only 11% of those medication studies compared medications with nonmedical or behavioral interventions.12

A robust CER enterprise, which encompasses policy-making and funding agencies as well as the researchers and users of CER, must include a significant focus on observational research, not just randomized controlled trials; address a broad set of types of interventions, not just pharmaceuticals; and produce research that reflects all parts of the population being treated, not just a carefully selected sample. CER topics and study designs will likely continue to be investigator-influenced, but a more strategic approach to allocating CER funds has been proposed, focused on addressing patients', clinicians', and decision makers' needs.9 This targeted funding will likely call for investigators to collaborate on funding proposals and partner with the care delivery system, including providers and patients, to conduct studies in “real-world” settings.

To compete effectively for CER funding, AHCs will require some effective degree of integration (virtual or actual) of physicians, hospital, and outpatient facilities, including postacute care settings; an effective information system that bridges these elements of the care system (discussed below); and a robust health services research capacity, preferably based in the academic enterprise (usually a medical school or school of public health) as well as in the hospital. Such infrastructure is especially critical to the design and support of research regarding new models of care, including care delivered by interdisciplinary teams, and transitions of care from hospital to posthospital settings. The AHRQ has invested in building this research capacity in numerous AHCs through its Effective Health Care Program.13

Back to Top | Article Outline

Human and scientific capital

The health services research workforce swelled from about 5,000 in 1995 to more than 13,000 in 2009.14 CER, however, requires health services and clinical researchers to develop new skills and capabilities. The limited number of CER-trained researchers, an ambitious national research agenda, and emerging funding opportunities for researcher development together create a significant opportunity for AHCs to expand training. The most important considerations in training the next generation of researchers are providing a strong methodological foundation and ensuring a multidisciplinary approach. To conduct CER and implement findings, researchers must understand diverse research methodologies for real-world studies, health information technology (HIT), health economics, and implementation science, and they must possess management skills.

Because there has been a gap in human capital, and funding for CER infrastructure has only recently increased, the validation and use of methodologies for conducting CER and evaluating its translation and impact are still in their infancy. If the CER enterprise is to be successful in delivering real-world results to patients and decision makers, then more efficient, generalizable trial designs must be debated, tested, and employed.15 AHCs need to build scientific capital by engaging in this dialogue and identifying opportunities to conduct trials that incorporate designs appropriate for CER.

Back to Top | Article Outline

Data infrastructure

To support CER, the federal government is making data infrastructure development a priority. Data infrastructure represents the largest investment from the $400 million in ARRA CER funds allocated to the Secretary of Health and Human Services. These funds are being invested in projects such as a multipayer claims database, new patient registries, creation of distributed clinical data networks, and development of community health networks.6 These investments will produce resources to support CER at AHCs through new datasets, improved data collection, and strengthened collaboration. The tools will only be effective, however, to the extent that they are validated and adopted by the academic and clinical communities.

AHCs also have significant opportunities to influence the development of these tools. The solicitation for the implementation of a multipayer claims database, for example, requires that the selected contractor engage health services researchers from AHCs to ensure that the tools and resources it produces are of value to the research community.16 In addition, AHCs are often central players in regional clinical data networks and leaders in HIT, so they can be at the vanguard of developing systems that collect the clinical data necessary to conduct robust CER as part of the frontline care delivery process.17 In addition, the electronic health record (EHR) systems that already exist in most AHCs can support new analysis for CER as well as the monitoring and management of the implementation of CER findings. A critical factor that will influence an AHC's capacity to support CER is the degree to which its HIT system is integrated across different providers (hospitals and physicians) and different settings (e.g., in- and outpatient, private office and clinic, AHC- and community-based facilities, long-term care and rehabilitation facilities, home care programs). AHCs should thus work to expand their HIT infrastructure as needed and ensure compatibility with other settings in their local health systems to build capacity for CER.

Back to Top | Article Outline

Translation and dissemination

AHCs offer the ideal setting for translation and dissemination of CER because their faculty, staff, and trainees conduct research side-by-side with patient care, which allows findings from CER projects to be rapidly incorporated into care delivery. Some AHCs, such as those with Clinical and Translational Science Awards (CTSAs), have established centers with a specific focus on research translation.18 The same or similar infrastructure could be employed to translate CER into patient care. Many tertiary care centers enjoy a broad referral base that also provides them with a large patient volume, thereby increasing the number of patients who may have access to the CER resources they offer.

As the CER enterprise and evidence base grow, opportunities to apply findings to hospital operations will appear. Although integrating CER into Medicare coverage decisions requires a transparent, iterative process that includes a period for public comment, hospital management teams could independently streamline their use of drugs within certain classes, medical devices, and therapeutic and diagnostic procedures based on CER findings if doing so were in their patients' best interests. Incorporating CER findings into practice guidelines, the EHR provider interface, and hospital procedures can also boost adoption in clinical practice.19 Cincinnati Children's, for example, is using EHRs to monitor physician performance on metrics that are the focus of quality improvement initiatives, a process that has begun to be applied to efforts borne from CER evidence.

As the public becomes more aware of the importance of CER, it is likely that AHCs will market their fidelity to CER-related evidence and their capacity to conduct CER. This approach has already been adopted by many hospitals that have received high rankings from U.S. News and World Report as well as those that have achieved “Magnet” status through the American Nurses' Credentialing Center for their excellent nursing programs. Most AHCs find themselves in an increasingly competitive marketplace. Designation as a “CER Center of Excellence,” either by the AHC itself or by an independent group, seems likely to appear in marketing campaigns in the near future.

Back to Top | Article Outline

Adapting Opportunities for Each Institution

As described above, AHCs have opportunities to benefit from and shape CER across four dimensions. However, to capture the advantages of CER at the institutional level, each AHC must identify where and how it intends to benefit from and contribute to the CER enterprise in research, education, and clinical care, and how it can adapt CER findings and resources to meet local needs.20 CER can inform multiple fields, ranging from medicine to health policy and education, and multiple functions, including clinical care, quality, purchasing, community outreach, and education. The CER enterprise also demands that institutions inform national priorities and that they share their successes and failures—in research, education, and translation—through peer-reviewed publications. To adapt the national CER agenda, policy trends, and funding opportunities to the local level, AHCs need leaders and faculty who have cross-functional skills, can integrate research and frontline improvement efforts, are able to adopt system-level thinking, and can form connections across AHC divisions and with other policy and research communities.

To address the cross-functional and pan-institutional demands and opportunities posed by CER, we propose that AHCs adopt an organizational approach (Figure 2). A number of AHCs have located their CER enterprise in health services research and policy departments with responsibility for clinical care quality. At Cincinnati Children's and Mount Sinai, the head of the academic clinical effectiveness department (Cincinnati Children's) or the health policy department (Mount Sinai) also has a title and role as a senior executive for hospital quality. We describe these AHCs' CER efforts below.

Figure 2

Figure 2

Back to Top | Article Outline

Cincinnati Children's

At Cincinnati Children's, the Division of Health Policy and Clinical Effectiveness actively develops evidence and translates it into practice or policy at the institutional, community, and national levels. The division was established in the mid-1990s and increased its ability to achieve system-level improvement following its receipt of a Robert Wood Johnson Foundation Pursuing Perfection grant in 2001. During 2007 to 2010, a common EHR was implemented across the system, and grant funding, including for CER, grew rapidly. One local clinical example of implementing evidence is the use of the EHR to detect adverse drug events in children in close to “real time,” analyze the causes and underlying evidence, and then put into place improvement teams; as a result, outcomes for children have dramatically improved. In the community, Cincinnati Children's is committed to improving outcomes. The AHC has instituted programs comparing different interventions that have thus far reduced infant mortality and premature birth in the surrounding region by over 15%, without regard for the potential negative economic effects on the institution due to lower neonatal ICU care volume.

Nationally, investigators lead improvement networks focused on patient populations with inflammatory bowel disease and cardiac disease and use social networking and other innovations to engage patients and their families in decision making and improving outcomes. From a policy perspective, the institution actively analyzes observational health services research data (e.g., Medicaid datasets) to describe state and national trends and potential policy levers. The organizational structure allows the division to be involved in identifying a local opportunity related to hospital quality of care and national CER priorities, develop and implement a solution, evaluate the success of the solution using appropriate methodologies, and ensure that the results contribute to the national dialogue. The division's success in such efforts led to the September 2010 launch of the James M. Anderson Center for Health Systems Excellence, which will enable Cincinnati Children's to increase its institutional, community, and national efforts to achieve step changes in improving outcomes and transforming health systems.

To embed the commitment to CER and quality improvement in the education and clinical care mission, Cincinnati Children's has trained numerous faculty, staff, and hospital administrators, including CEOs and other senior leaders, in improvement methods and testing interventions. Faculty and fellows also have opportunities to receive health services and CER training. The AHC has formed cross-functional improvement teams to focus on implementing evidence in the clinical arena. These teams typically include a member from the Anderson Center and often include researchers, clinicians, patients or patients' family members, and students. This system provides a means for implementing and refining CER results and serves as a forum for educating students, physicians, staff, and patients about CER and implementation science. Cincinnati Children's has much to learn and improve on, but the above examples demonstrate the potential for CER to transform health and health care by engaging constituents from across the AHC, partnering with stakeholders outside the AHC's walls, and weaving CER into the fabric of care delivery.

Back to Top | Article Outline

Mount Sinai

More than a decade ago, Mount Sinai created two parallel entities to partner in CER and its application. The first was to establish at Mount Sinai School of Medicine of New York University the Department of Health Policy (now the Department of Health Evidence and Policy), which houses a cadre of health services researchers and policy experts. The second was the simultaneous development of a major health care quality effort at the medical center, under the direction of a senior vice president for quality. At the outset, that senior vice president also served as the chair of the medical school's health policy department. This arrangement fostered close collaboration and integration. As the two programs grew, the AHC leadership decided to separate the two roles, but the close collaboration and cooperation that persist position Mount Sinai very well to respond to the opportunities and challenges of CER, as well as health care reform in general. The prior establishment of an effective and well-recognized program focusing on evidence-based research was instrumental in facilitating Mount Sinai's development of its CTSA, known as Conduits–the Institutes for Translational Sciences.

Back to Top | Article Outline

Expanding the reach of CER

Although AHCs often serve diverse populations, expanding their reach in surrounding communities could dramatically improve their ability to involve a broader set of priority populations in CER studies and strengthen dissemination and translation efforts. Cincinnati Children's has providers in some community hospitals and outpatient facilities throughout the region, which allows the AHC to spread interventions and conduct research in real-world settings in keeping with its strategic goal to expand its scope further into the community. Such community relationships help AHCs achieve successes, such as the decreased infant mortality noted above. Communities can also provide critical feedback to AHCs about CER needs and the decisions the CER enterprise should address at the community and national levels. In terms of conducting CER, networks of providers in the community are critical for conducting research on populations that are more generalizable and include sufficient numbers of priority populations such as racial and ethnic minorities whenever possible.

Back to Top | Article Outline

Shaping the national debate

To be successful, the CER enterprise must be shaped by clinicians, patients, and everyone involved in health care decisions. Participation by AHCs is crucial because they employ thousands of clinicians, provide care for patients (often when they are most in need), and are the academic home for CER researchers. AHCs should be active in providing leadership and input to the Patient-Centered Outcomes Research Institute, which Section 6301 of the PPACA established to set priorities for and fund CER.4 AHCs should also actively solicit input and feedback from frontline clinicians and patients to identify the most pressing questions that need answers and how best to approach answering these questions. In addition, AHCs can be a conduit for information that policy makers and funding agencies require to make effective decisions on priorities. Finally, AHCs can lead the research enterprise by rewarding collaborative researchers who are addressing CER needs identified by decision makers, which may preclude their pursuing the independent investigator path, with promotion and other academic incentives.

Back to Top | Article Outline

Barriers to Change

Institutions that wish to take full advantage of CER opportunities by establishing a cross-functional clinical effectiveness and health policy center or division face three primary barriers. The first is shifting the culture toward greater interdivision collaboration and increased acceptance of observational analysis to inform practice. A cross-functional clinical effectiveness division can only succeed in its quality improvement mission if it has productive, collaborative relationships with other clinical divisions. Whereas clinical care improvement and research have historically occupied separate silos at major AHCs, the advent of cross-functional divisions at Cincinnati Children's and Mount Sinai may signal a new trend and serve as catalysts for collaboration elsewhere. Additionally, although randomized clinical trials are likely to remain the “gold standard” of clinical information, clinicians and administrators need to identify appropriate ways to incorporate other kinds of data and analysis into decision making. Observational studies have the potential to be powerful tools for real-time analysis of practice patterns and outcomes at AHCs. In both of these cultural shifts, the support of senior institutional and division leadership as well as shared goals and accountability are critical for success.

The second barrier is identifying appropriate talent to lead the clinical effectiveness division. CER is intended to shape real-world clinical decisions. As such, clinical effectiveness leaders must be able to identify opportunities for improvement, mobilize resources to analyze and address them, and manage execution leading to change in care and improved performance. The ideal candidates to take charge of such efforts would have experience leading clinical teams, expertise in research, and a track record of developing and implementing change. They would also have significant exposure to networks outside of their own institution, especially in policy arenas, to enhance their institution's capability to shape CER at the national level.

The third barrier is the ability to deploy enabling technology. Most AHCs use EHRs, but the ease with which each system can be used for research and improvement efforts varies greatly. An EHR system that can seamlessly support research functionality and performance measurement is critical to realize the potential impact of a clinical effectiveness center or division. The use of EHRs in the communities surrounding AHCs is often variable and may involve different systems. If community partners are to be research collaborators, interoperable HIT systems are often necessary.

Back to Top | Article Outline


AHCs have tremendous opportunities to set priorities for and benefit from the CER enterprise at the local and national levels. The public policy momentum and increased public funding make now the time for AHCs to invest in CER at the institution level. AHCs are uniquely positioned to help shape the CER so that it will have the greatest possible effect on patient care, given the alignment of their three-pronged mission with CER's four dimensions of research, education, infrastructure, and translation. An AHC's capacity to conduct and implement CER will also affect its ability to develop an effective accountable care organization (ACO) because high-quality, evidence-based care is central to the role of ACOs, which are a component of the PPACA.

Becoming “CER-ready,” however, will require innovation in the way that AHCs do business (Figure 2). They must each establish a central cross-functional and pan-institutional organization that can help adapt CER opportunities to the institution and work across departments and divisions to ensure that CER receives appropriate focus. They must also maintain strong bidirectional communication with the national CER enterprise, to stay abreast of national priorities and funding opportunities, as well as to share the results of observational and implementation research at their own institution and to inform future priorities.

Federal policy and health care reform legislation have increased national attention paid to CER, but progress in research and implementation of changes in patient care that improve quality must be enhanced at the regional and local levels. AHCs will play a critical role in conducting CER, training the workforce, developing the data infrastructure, implementing CER findings, and informing the CER enterprise going forward. The CER movement presents challenges and, more important, opportunities for AHCs.

Back to Top | Article Outline


The authors would like to acknowledge the leadership of Dr. Uma Kotagal (Cincinnati Children's) and Dr. Mark Chassin (Mount Sinai) as well as the faculty and staff at Cincinnati Children's and Mount Sinai who served as examples to be cited in this article.

Back to Top | Article Outline


Dr. Conway is supported in part by the Agency for Healthcare Research and Quality U18 HS 016957 and R01HS019862-01, the Department of Health and Human Services HHSP233201999162A, and the Child Health Corporation of America as part of the Pediatric Research in Inpatient Settings network. Dr. Rowe is supported by the John D. and Catherine T. MacArthur Foundation. None of these funding sources contributed to or influenced this article.

Back to Top | Article Outline

Other disclosures:


Back to Top | Article Outline

Ethical approval:

Not applicable.

Back to Top | Article Outline


The views expressed in this manuscript represent the authors' views and do not represent official policy or opinions of the U.S. Department of Health and Human Services, Columbia University, or Cincinnati Children's Hospital.

Back to Top | Article Outline


1Dow Jones. Factiva. Accessed February 23, 2011.
3American Recovery and Reinvestment Act of 2009. Pub L No. 111-5, 123 Stat. 115.
4The Patient Protection and Affordable Care Act of 2010. Pub L No. 111-148, 124 Stat. 119.
5U.S. Department of Health and Human Services. Budget of the United States Government, Fiscal Year 2011. Accessed February 15, 2011.
6Federal Coordinating Council for Comparative Effectiveness Research. Report to the President and the Congress. Washington, DC: U.S. Department of Health and Human Services; June 2009. Accessed February 15, 2011.
7U.S. Department of Health and Human Services. Comparative Effectiveness Research Funding. Accessed February 15, 2011.
8Institute of Medicine. Initial National Priorities for Comparative Effectiveness Research. Washington, DC: National Academies Press; 2009.
9VanLare JM, Conway PH, Sox HC. Five next steps for a new national program for comparative effectiveness research. N Engl J Med. 2010;362:970–973.
10Conway PH, Clancy C. Charting a path from comparative effectiveness funding to improved patient-centered health care. JAMA. 2010;303:985–986.
11The Lewin Group. Federal Coordinating Council for Comparative Effectiveness Research: CER Inventory and Strategic Framework Final Report. Washington, DC: The Lewin Group, LLC; 2009.
12Hochman M, McCormick D. Characteristics of published comparative effectiveness studies of medications. JAMA. 2010;303:951–958.
13Agency for Healthcare Research and Quality, Effective Health Care Program. Comparative Effectiveness Research Grant and ARRA Awards. Accessed February 21, 2011.
14Ricketts TC. Preparing the health services research workforce. Health Serv Res. 2009;44:2227–2241.
15Luce BR, Kramer JM, Goodman SN, et al. Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann Intern Med. 2009;151:206–209.
16U.S. Department of Health and Human Services, Centers for Medicare and Medicaid Services. RECOVERY–Multi-Payor Claims Database. Solicitation no. RFQ-MPCD-2010-DRCG03. Accessed February 21, 2011.
17Conway PH, Clancy C. Transformation of health care at the frontline. JAMA. 2009;301:763–765.
18Clinical and Translational Science Awards. About CTSAs. Accessed February 15, 2011.
19Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
20Berwick DM. Disseminating innovations in health care. JAMA. 2003;289:1969–1975.
© 2011 Association of American Medical Colleges