Secondary Logo

Share this article on:

Characteristics of Early Recipients of Patient-Centered Outcomes Research Institute Funding

Mazur, Stephany; Bazemore, Andrew MD; Merenstein, Daniel MD

doi: 10.1097/ACM.0000000000001115
Articles

The Patient Protection and Affordable Care Act (ACA) is grounded in the goals of increasing access, improving quality, and reducing cost in the U.S. health care system. The ACA established the Patient-Centered Outcomes Research Institute (PCORI) to help accomplish these goals through patient-focused research. PCORI has a different charge than its federally supported counterpart, the National Institutes of Health (NIH)—to fund research that ultimately helps patients make better-informed health care decisions. The authors examined characteristics of the recipients and settings of the first six rounds of PCORI funding and differentiated PCORI and NIH funding patterns to analyze the extent to which PCORI is accomplishing the goals set out by the ACA.

The authors performed a retrospective review of publicly available datasets, supplemented by a short questionnaire to funded PCORI principal investigators (PIs). The authors analyzed PCORI’s first six funding cycles (2011–2014) and data on NIH funding patterns (2000–2013) to determine whether PCORI and NIH funding patterns differed by investigator, department, and institution, and whether PCORI had funded research in primary care settings. The authors found that PCORI is funding a more diverse cadre of PIs and biomedical departments than is NIH, but not a greater diversity of institutions, and that less than one-third of PCORI studies involve or are relevant to primary care—the largest patient care platform in the United States. As PCORI looks to be refunded, it is important that research funding is further evaluated and publicly acknowledged to assess whether goals are being achieved.

S. Mazur is a third-year student, Georgetown University School of Medicine, Washington, DC.

A. Bazemore is director, Robert Graham Center, American Academy of Family Physicians, Washington, DC.

D. Merenstein is research division director, Department of Family Medicine, Georgetown University Medical Center, Washington, DC.

Editor’s Note: This New Conversations contribution is part of the journal’s ongoing conversation on the present and future impacts of current health care reform efforts on medical education, health care delivery, and research at academic health centers, and the effects such reforms might have on the overall health of communities.

A Commentary by J.V. Selby and J.R. Slutsky appears on pages 453–454.

To read other New Conversations pieces and to contribute, browse the New Conversations collection on the journal’s Web site (http://journals.lww.com/academicmedicine/pages/collectiondetails.aspx?TopicalCollectionId=43), follow the discussion on AM Rounds (academicmedicineblog.org) and Twitter (@AcadMedJournal using #AcMedConversations), and submit manuscripts using the article type “New Conversations” (see Dr. Sklar’s January 2015 editorial for submission instructions and for more information about this feature).

Funding/Support: The authors would like to thank the Georgetown University School of Medicine Office of Student Research for their financial support of this project.

Other disclosures: None reported.

Ethical approval: This study was reviewed by the institutional review board of the American Academy of Family Physicians and determined not to constitute human subjects research.

Correspondence should be addressed to Daniel J. Merenstein, Georgetown University Medical Center, 4000 Reservoir Rd., NW, Building D 240, Washington, DC 20007; telephone: (202) 687-2745; e-mail: djm23@georgetown.edu.

The National Institutes of Health (NIH), which has been the foundation of the U.S. biomedical research enterprise since its establishment in 1887, has a stated mission to “seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.”1 However, as U.S. health care shifts to become more individualized and patient centered, many have noted an increasing need for patient-centered research and delivery of care.2–4

The Patient Protection and Affordable Care Act (ACA) established the Patient-Centered Outcomes Research Institute (PCORI) as an entity to help usher the United States into the era of more patient-centered research. Launched in 2010, the mission of PCORI is to “[help] people make informed healthcare decisions, and [improve] healthcare delivery and outcomes, by producing and promoting high-integrity, evidence-based information that comes from research guided by patients, caregivers, and the broader healthcare community.”5 The vision of PCORI is that “patients and the public have information they can use to make decisions that reflect their desired health outcomes.”5 As PCORI has stated, “The essence of the PCORI definition of patient-centered outcomes research is the evaluation of questions and outcomes meaningful and important to patients and caregivers.”6 PCORI’s purpose thus differs in important ways from that of NIH: PCORI’s mission explicitly aims to help patients to make health care decisions, while the mission of NIH is rooted in basic scientific discovery. While NIH and PCORI both aim to improve health care, their roles in accomplishing this aim greatly differ, and it follows that these two institutions might distribute funding distinctly.

Given its novel approach and unique position in the ecology of U.S. health research funders, one would posit that PCORI might differ in its funding patterns from the well-established NIH in important ways, such as which topics, settings, investigators, and institutions it funds.

We set out to examine how PCORI awards are being distributed by evaluating publicly available data on the early rounds of PCORI funding. As the missions of PCORI and NIH are different, we expected to find that PCORI would fund different types of educational degree of principal investigators (PIs), biomedical departments, and institutions than are funded by NIH. Although is it difficult to analyze the entirety of NIH’s annual multibillion dollar budget, distributed funding can be examined by some key measures that can serve as points of comparison.

As patients seek care more frequently in the primary care setting than in any other platform of U.S. health care—accounting for fully 50% of all doctor’s visits—there is a particular need for meaningful patient-centered research in the field of primary care, which is expanding under the ACA.7–11 As we examined PCORI funding practices, we wanted to see whether PCORI would fund a commensurate share of research relevant to or occurring in the primary care setting.

Back to Top | Article Outline

Examining the Data

PCORI funding data

This project was reviewed by the institutional review board of the American Academy of Family Physicians and determined not to constitute human subjects research. The publicly accessible PCORI Web site (pcori.org) was used to access information on PCORI-funded awards. The following funding cycles were assessed: Pilot Projects, Cycle I, Cycle II, Cycle III, August 2013, and July 2014 research funding awards, which covers PCORI funding between 2011 and 2014. The Pipeline to Proposal funding awards were not included because “these are not research awards but are designed to better integrate patients and other stakeholders into the research process.”12 In sum, 318 PCORI-funded studies were analyzed.

We determined the academic degree of any PCORI-funded study’s PI using publicly available Internet search engines. In this process, we captured the highest academic degree of the PI, any additional degrees higher than a bachelor’s degree, and the PI’s biomedical department. Biomedical department refers to the field that the PI is trained in or practices in, such as epidemiology, pediatrics, or biostatistics. If more than one department was listed, we recorded the department in which the PI held the most senior position.

The site of participant recruitment for each PCORI-funded study was determined by two means. First, we contacted each of the PIs. Of the 318 PIs, 299 PIs were e-mailed to determine the recruitment site of the patients or participants for their study. Of the 19 other PIs, 17 were not e-mailed because their study dealt with statistical methods or other methods that did not involve patients. The e-mail addresses of the remaining two PIs could not be found. The PIs were asked: “Where were the majority of your patients for this study recruited from (i.e., hospital, community clinic, primary care clinic, subspecialty clinic, online, etc.)? Please be as descriptive as possible.” They were also asked, “Does this project involve primary care sites or primary care patients?” If the PI did not respond to the e-mail, the patient recruitment site was obtained from the abstract of the study on PCORI’s Web site or from clinicaltrials.gov. If more than one recruitment site was stated, all of them were included. An “other” category was made to include the recruitment sites senior living community, school, hospice, HMO, ACO, university, wellness center, and nursing homes.

Additionally, two investigators (D.M. and S.M.) read through all of the funded abstracts separately, and each rated the study as either pertaining to primary care or not pertaining to primary care, based on whether the study’s abstract on pcori.org specifically mentioned the involvement of primary care patients or primary care site in the study design. This step was included to look at primary care funding for comparison with the responses the PIs e-mailed. Once the two team members rated the 299 abstracts individually, the ratings were compared.

Back to Top | Article Outline

NIH funding data

As annual NIH funding is expansive, we relied on previously published studies to identify some characteristics of NIH funding patterns, namely educational background of PIs and biomedical department funded. Ginther and colleagues13 reported comprehensive data regarding the educational background of R01 type 1–funded NIH grantees from 2000 to 2006.13 The biomedical department of the NIH-funded PIs was obtained through the Blue Ridge Institute for Medical Research, which used the NIH Research Portfolio Online Reporting Tool (RePORT) to access the departments funded for fiscal year 2013 and included all grantees except for American Recovery and Reinvestment awards.14 We also gathered data about which institutions received funding in fiscal year 2013 from the NIH RePORT database.

Back to Top | Article Outline

Comparing PCORI and NIH funding data

We calculated the percentage of NIH funding awarded to each department; in fiscal year 2013, NIH funded 44 categorized departments, and the top 40 most-funded NIH departments by percentage were compared with all PCORI-funded departments. The four least-funded departments (physics, nutrition, chemistry, and dentistry) were excluded for ease of data analysis.

We compared funding awarded by NIH and PCORI to institutions including universities, medical centers, and private and not-for-profit research groups. To compare the percentage of funding awarded to various institutions from PCORI and NIH, total funding to all institutions from NIH was found by searching the NIH RePORT Web site for Awards by Location for fiscal year 2013.15 The percentage of total funding to each institution was calculated. NIH-funded institutions, based on percentage of total funding, were broken down into groups of 10 to simplify data management and comparison (i.e., institutions ranking 1–10 together, institutions ranking 11–20 together, etc.). The total percentage of PCORI funding to these same 40 institutions was calculated. PCORI funding to each of these groups of 10 was calculated and compared with the percentage of NIH funding to these same groups of 10. See Table 1 for the full listing of institutions.

Table 1

Table 1

Back to Top | Article Outline

Characteristics of NIH- and PCORI-Funded Awards

Academic degree of PCORI- and NIH-funded PIs

Approximately 61% (193) of PCORI-funded PIs hold an MD degree (see Figure 1). Additionally, 54% (172) of PCORI-funded PIs hold a diversity of additional master’s degrees or non-MD/non-PhD doctoral degrees. Historically, 67% (18,528) of NIH’s R01 type 1 grants are awarded to PIs with a PhD, compared with 38% (122) of PCORI’s funding awarded to PhD-holding PIs.12,16–18

Figure 1

Figure 1

Back to Top | Article Outline

Biomedical department of PCORI- and NIH-funded PIs

Others have observed that 70% of NIH funding is awarded to the top 10 most-funded departments, and 2% of NIH funding goes to the 10 least-funded departments.14 PCORI’s funding is less concentrated to the top 10 NIH-funded departments, with 40% of funding going to the top 10 and 26% of funding going toward departments in the bottom bracket of NIH-funded departments (see Figure 2).

Figure 2

Figure 2

Back to Top | Article Outline

Institutional affiliation of PCORI- and NIH-funded PIs

Based on publicly available information, PCORI and NIH both award similar proportions of funding to the top 40 NIH-funded institutions. NIH awards 50% of its funding to its top 40 institutions. PCORI awards 51% of its funding to the same 40 institutions (see Figure 3).

Figure 3

Figure 3

Back to Top | Article Outline

Recruitment site for PCORI studies

Of the 299 PIs e-mailed, responses to the question of participant recruitment site were received from 213, representing a 71% response rate. The recruitment sites of 59 studies were excluded because they did not involve patient participants (n = 24) or they could not be determined (n = 35). PCORI-funded PIs recruited patients from eight main types of recruitment sites. The number of studies recruiting participants from subspecialty clinics was slightly greater (n = 83) than the number recruiting from primary care clinics (n = 81). Hospitals were used as recruitment sites more frequently than community settings, the Internet, or databases (see Figure 4).

Figure 4

Figure 4

Back to Top | Article Outline

Primary care involvement in PCORI studies

Based on information gathered through e-mail responses from PIs and supplemented by clinicaltrials.gov or the PCORI Web site, 32% (n = 96) of the 299 studies involved primary care patients or primary care sites. However, when the study abstracts were reviewed by D.M and S.M., 19% (n = 57) of studies were found to deal with primary care, defined as a study design involving the participation of a primary care provider or a primary care site. Of the 299 studies reviewed, there were 11 cases of disagreement among the team members, and for these 11, the two team members consulted and came to an agreement on each. The Cohen kappa score was 0.93 for D.M. and 0.95 for S.M. (data not shown).

Back to Top | Article Outline

Discussion

Our analysis reveals notable early differences in the PIs and biomedical departments being funded by PCORI, when compared with NIH, though similar institutions seem to be receiving the bulk of the funding. The U.S. research funding landscape remains dominated by the NIH,19 whose annual budget is nearly $30 billion. Prior reports have explored the demographics, including race, gender, and age of NIH PIs, provoking NIH efforts to diversify their distribution of funding.16,20–22 According to three previous studies, NIH heavily funds PIs holding a PhD: Two-thirds of funded R01 type 1 PIs hold a PhD degree, compared with one-third holding an MD degree.13,17,18 When compared with NIH, PCORI is funding many fewer PhDs (38%), more MDs, and PIs with more varied backgrounds, which is likely supporting their mission of affecting the patient care experience as more MDs, social workers, and others involved directly in patient care are receiving funding.

In addition to funding PIs of different educational backgrounds than NIH-funded PIs, PCORI is also funding PIs of different biomedical departments. NIH awards the majority of its funding to 10 types of departments, and just 2% of its funding goes to its least-funded types of departments. PCORI sends a large bulk of its funding (25%) to the bottom 10 funded NIH departments. This represents a major point of success for PCORI as it helps diversify the U.S. research landscape. One example of PCORI’s diversification of the research landscape is that PCORI specifically has outlined that one of its priorities is to fund databases and research on rare diseases,23 which have historically received little funding from major research institutions. By funding different PIs and biomedical departments than NIH, PCORI is supporting researchers, departments, and topics that were previously less funded.

Despite the fact that PCORI is funding a more diverse cadre of PIs and departments than NIH, most of its funding is going to similar institutions as the ones that receive the bulk of NIH funding. In terms of funding to institutions, both NIH and PCORI distribute 19% of their funding to the top 10 NIH-funded institutions, and both distribute about 50% of their funding to the top 40 NIH-funded institutions. The tendency of research funding to aggregate to institutions with similarly endowed research infrastructure and experienced investigators is not surprising, nor necessarily concerning. It does suggest that newer institutions and those with lower success in attracting NIH funding—many of them founded with principles of community and patient-centered training in their missions—may also have lower success in receiving PCORI funding. Whether this risks homogeneity in research methods and perspectives, and in areas and research populations studied, and whether such homogeneity matters, is yet to be determined. Evaluation of these risks will be important, as will evaluation of the outcomes and impact of these studies as they are completed in the future. PCORI may also continue to explore ways to gather meritorious applications from disadvantaged institutions whose perspectives, missions, populations served, and research approaches may differ from those found in the traditional NIH research powerhouses.

Half of all health care in the United States is delivered in primary care settings, a finding consistently demonstrated since the 1960s.10,11 However, there is a paucity of NIH-funded research taking place in these settings.24,25 For example, only 0.2% of all NIH funding from 2002 to 2006 was awarded to PIs in departments of family medicine.25 With a majority of ambulatory care occurring in primary care settings, one might have expected a commensurate share of PCORI-funded research to be conducted in such settings. Improving care in these frontline venues, which are undergoing wholesale transformation into patient-centered medical homes (PCMHs), represents an excellent opportunity to influence the patient-centered outcomes central to PCORI’s vision. The PCMHs at the center of practice-based research networks are uniquely positioned to fulfill PCORI’s mission, to help patients make informed health care decisions based on evidence-based information. However, according to self-report by PCORI-funded PIs, only 32% of PCORI-funded studies involve any primary care sites or patient populations. Our analysis of primary care involvement for the same studies found that only 19% of the studies involved primary care. Whether 19% or 32%, the proportion of PCORI funding for research relevant to or occurring in primary care is lower than we expected to find. With increasing need for high-quality, cost-effective primary care under the ACA, an increase in funding to primary care research may need to be considered. PCORI may wish to explore whether primary care patients and settings are adequately represented on advisory and research boards, as it appears that primary care has not been a priority of early PCORI funding.

Our analysis has some limitations in its ability to assess PCORI’s funding patterns. Web-based information gathering offers limited ability to assess whether data gathered is accurate or current; however, we cross-referenced our information where possible. Another limitation is that analysis of the educational background of NIH PIs was limited to R01 type 1 grantees because of unavailability of further information publicly. Information gathered via survey of study PIs is potentially subject to both reporting and recall bias, and indeed, PIs reported 13% higher primary care involvement than did our investigator-blinded review of study abstracts. Detailed analysis of the outcomes of the PCORI-funded studies was not possible because of a lack of publicly available publications at this time.

Overall, PCORI’s pattern of funding appears to differ in important ways from that of NIH. PCORI is proportionally funding more MDs than is NIH, and is funding a greater diversity of biomedical departments, but is still funding many of the same institutions as NIH. PCORI’s mission is to focus on patient-driven research; this effort can be strengthened by funding studies that involve primary care, where half of health care is accessed. The ACA mandated in 2010 that PCORI receive funding through FY2019. As PCORI is now halfway into its established life, it is important that PCORI continue to evaluate its funding patterns and assess if it is meeting its missions.

Acknowledgments: The authors acknowledge Cayla Fappiano for her careful review of this manuscript.

Back to Top | Article Outline

References

1. National Institutes of Health. Mission and goals. http://www.nih.gov/about/mission.htm. Accessed December 2, 2015
2. Weinberger SE, Johnson BH, Ness DL.. Patient- and family-centered medical education: The next revolution in medical education? Ann Intern Med. 2014;161:73–75
3. Laine C, Davidoff F.. Patient-centered medicine. A professional evolution. JAMA. 1996;275:152–156
4. Smith M, Saunders R, Stuckhardt L, McGinnis JM Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. 2014 Washington, DC National Academies Press
5. Patient-Centered Outcomes Research Institute. About us. http://www.pcori.org/about-us/mission-and-vision/. Accessed December 2, 2015
6. Frank L, Basch E, Selby JVPatient-Centered Outcomes Research Institute. . The PCORI perspective on patient-centered outcomes research. JAMA. 2014;312:1513–1514
7. Petterson SM, Liaw WR, Phillips RL Jr, Rabin DL, Meyers DS, Bazemore AW.. Projecting US primary care physician workforce needs: 2010–2025. Ann Fam Med. 2012;10:503–509
8. University of California, Los Angeles. UCLA social science data archive dataverse. https://dataverse.harvard.edu/dataverse/ssda_ucla. Accessed December 9, 2015
9. Starfield B, Shi L, Macinko J.. Contribution of primary care to health systems and health. Milbank Q. 2005;83:457–502
10. Green LA, Fryer GE Jr, Yawn BP, Lanier D, Dovey SM.. The ecology of medical care revisited. N Engl J Med. 2001;344:2021–2025
11. White KL, Williams TF, Greenberg BG.. The ecology of medical care. N Engl J Med. 1961;265:885–892
13. Ginther DK, Haak LL, Schaffer WT, Kington R.. Are race, ethnicity, and medical school affiliation associated with NIH R01 type 1 award probability for physician investigators? Acad Med. 2012;87:1516–1524
14. Blue Ridge Institute for Medical Research. Ranking tables of NIH funding to US medical schools in 2006–2013. http://www.brimr.org/NIH_Awards/NIH_Awards.htm. Accessed December 2, 2015
15. National Institutes of Health Research Portfolio Online Reporting Tools (RePORT). NIH awards by location and organization. http://report.nih.gov/award/index.cfm. Accessed December 2, 2015
16. Ginther DK, Schaffer WT, Schnell J, et al. Race, ethnicity, and NIH research awards. Science. 2011;333:1015–1019
17. Jonisch AI, Kligerman S, Nagy E, Bhargavan M, Forman HP, Sunshine J.. What characterizes academic radiology departments that secure large amounts of external funding for research? Acad Radiol. 2006;13:1513–1516
18. Maguire MA, Gore JC.. The current state of NIH funding of research in diagnostic radiology at U.S. medical schools. J Am Coll Radiol. 2005;2:436–443
19. Owens B.. Mapping biomedical research in the USA. Lancet. 2014;384:11–14
20. Tabak LA, Collins FS.. Sociology. Weaving a richer tapestry in biomedical science. Science. 2011;333:940–941
21. Shavers VL, Fagan P, Lawrence D, et al. Barriers to racial/ethnic minority application and competition for NIH research funding. J Natl Med Assoc. 2005;97:1063–1077
22. National Institutes of Health. NIH guide for grants and contracts. http://grants.nih.gov/grants/guide/pa-files/PA-12–149.html. Accessed December 2, 2015
23. Patient-Centered Outcomes Research Institute. Advisory panel on rare disease. http://www.pcori.org/get-involved/join-advisory-panel/advisory-panel-rare-disease. Accessed December 9, 2015
24. Lucan SC, Phillips RL Jr, Bazemore AW.. Off the roadmap? Family medicine’s grant funding and committee representation at NIH. Ann Fam Med. 2008;6:534–542
25. Lucan SC, Barg FK, Bazemore AW, Phillips RL Jr. Family medicine, the NIH, and the medical-research roadmap: Perspectives from inside the NIH. Fam Med. 2009;41:188–196
© 2016 by the Association of American Medical Colleges