Share this article on:

A Logic Model for Community Engagement Within the Clinical and Translational Science Awards Consortium: Can We Measure What We Model?

Eder, Milton “Mickey PhD”; Carter-Edwards, Lori PhD; Hurd, Thelma C. MD; Rumala, Bernice B. PhD; Wallerstein, Nina DrPH

doi: 10.1097/ACM.0b013e31829b54ae
Perspectives

The Clinical and Translational Science Award (CTSA) initiative calls on academic health centers to engage communities around a clinical research relationship measured ultimately in terms of public health. Among a few initiatives involving university accountability for advancing public interests, a small CTSA workgroup devised a community engagement (CE) logic model that organizes common activities within a university–community infrastructure to facilitate CE in research. Whereas the model focuses on the range of institutional CE inputs, it purposefully does not include an approach for assessing how CE influences research implementation and outcomes. Rather, with communities and individuals beginning to transition into new research roles, this article emphasizes studying CE through specific relationship types and assessing how expanded research teams contribute to the full spectrum of translational science.

The authors propose a typology consisting of three relationship types—engagement, collaboration, and shared leadership—to provide a foundation for investigating community–academic contributions to the new CTSA research paradigm. The typology shifts attention from specific community–academic activities and, instead, encourages analyses focused on measuring the strength of relationships through variables like synergy and trust. The collaborative study of CE relationships will inform an understanding of CTSA infrastructure development in support of translational research and its goal, which is expressed in the logic model: better science, better answers, better population health.

Dr. Eder is director of research and evaluation, Access Community Health Network (a network of Federally Qualified Health Centers), Chicago, Illinois.

Dr. Carter-Edwards is deputy director of research and operations, Center for Health Promotion and Disease Prevention, and research associate professor of health behavior, Gillings School of Global Public Health, University of North Carolina, Chapel Hill, North Carolina. While preparing this article, she also was research assistant professor, Division of Community Health, Department of Community and Family Medicine, Duke School of Medicine, Durham, North Carolina.

Dr. Hurd is associate professor of surgery, Department of Surgery–Division of Surgical Oncology, University of Texas Health Science Center at San Antonio, San Antonio, Texas.

Dr. Rumala is visiting research scholar, Columbia University, Center for Health Equity and Urban Science Education, New York, New York. While preparing this article, she was a community engagement specialist, Center for Clinical and Translational Science, and codirector, SPARC Tri-Institutional Diversity Program, Rockefeller University, New York, New York.

Dr. Wallerstein is professor, Public Health Program, Department of Family and Community Medicine, and director, Center for Participatory Research, Office of Community Health, University of New Mexico, Albuquerque, New Mexico.

Acknowledgments: The authors wish to acknowledge and thank the following individuals who contributed to the development of this article: Syed Ahmed (Medical College of Wisconsin), Nancy Bennett (University of Rochester), Giselle Corbie-Smith (University of North Carolina), Carol Ferrans (University of Illinois–Chicago), Ellen Goldstein (University of California San Francisco), Karen Hacker (Harvard University), Jan Hogle (University of Wisconsin–Madison), Katherine Kaufer Christoffel (Northwestern University), Susan Lyons (University of Alabama at Birmingham), Donna Jo McCloskey (NIH/NINR), Lloyd Michener (Duke University), Mary Anne McDonald (Duke University), and Kathleen Stevens (University of Texas Health Science Center at San Antonio). With appreciation to all the Outcomes Workgroup participants for contributing to the exchange of ideas and to the Booz Allen Hamilton Project Managers (Jenna Goldstein, Allison Heiser, Valerie Kokai, Nicole Morales) and the Vanderbilt University CTSA Coordinating Center (Leslie Boone) for their indispensable assistance throughout the development of this article.

Funding/Support: This project has been funded in whole or in part with federal funds from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through the Clinical and Translational Science Awards Program (CTSA), part of the Roadmap Initiative, Re-Engineering the Clinical Research Enterprise. Support was provided by the Institute for Translational Medicine, University of Chicago (#UL1RR024999 and #UL1TR000430); the Center for Health Promotion and Disease Prevention, University of North Carolina (Centers for Disease Control Cooperative Agreement #U48-DP001944); the Duke Translational Medicine Institute, Duke University (#UL1RR024128); the Institute for Integration of Medicine and Science, University of Texas Health Science Center at San Antonio (#UL1RR025767); the Center for Clinical and Translational Science, Rockefeller University (#UL1RR024143); and the Clinical and Translational Science Center, University of New Mexico (#UL1RR031977). This article was also supported in part by Cooperative Agreement Number U48-DP001944 from the Centers for Disease Control and Prevention.

Other disclosures: The manuscript was approved by the CTSA Consortium Publications Committee.

Ethical approval: Not applicable.

Disclaimer: The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Correspondence should be addressed to Dr. Eder, Access Community Health Network, 600 W. Fulton St., Suite 200, Chicago, IL 60661; telephone: (312) 526-2116; e-mail: milton.eder@accesscommunityhealth.net.

The infrastructure within academic health centers (AHCs) to support community-engaged research (CEnR) has grown rapidly following the announcement of the Clinical and Translational Science Awards (CTSAs) in 2006.1 The director of the National Institutes of Health (NIH) at that time, Elias Zerhouni, called the CTSAs “the first systematic change in our approach to clinical research in 50 years.”2

Back to Top | Article Outline

Research and Public Accountability

With an emphasis on realigning institutional relationships and organizing new community and clinical networks, the NIH launched the CTSA program under the auspices of the National Center for Research Resources and then transitioned it in 2012 to the recently formed National Center for Advancing Translational Sciences (NCATS). The CTSA program supports an array of assets to streamline science, transform training environments, and improve the conduct, quality, and dissemination of clinical and translational research. The CTSA Roadmap initiative is developing translational research as a means to more readily and systematically bridge stages within clinical research discovery processes.3,4

Each CTSA institution is expected to participate actively in the CTSA national consortium. The consortium, comprising 60 AHCs, included among its strategic goals a commitment to developing and strengthening community–academic research partnerships to enhance the capacity of AHCs to improve the health of communities and the nation.5 The challenges faced by the consortium for fostering cross-institutional collaboration and community engagement (CE) as forms of team science are vast and include the voluntary nature of participation in consortium activities and the fact that the efforts of those individuals who work within consortium teams to advance science are not always or uniformly recognized by consortium, institutional, and community leaders. Despite the challenges, and amidst evolving NIH, consortium, and institutional aims, researchers and evaluators pursue and assess CE and CEnR.

The CTSA adoption of CE aligns with other projects that address the public accountability of universities to their surrounding communities.6–9 Academic–community research partnerships that use scientific methods to produce socially responsible and community-beneficial knowledge have encountered both facilitators of and challenges to these partnerships. One of these challenges is assessing and evaluating the contributions of these partnerships.9–11 While efforts to develop and institute evaluation strategies for translational science are emerging,12,13 the CTSA program commitment to bidirectionality (coindependent variables) as a characteristic of translational science has complicated evaluation of CE. CTSA evaluators, initially focused on how internal institutional reorganization supported improvements in academic research processes, have only recently begun to examine whether and how community–academic interactions contribute to research and knowledge production.14

Back to Top | Article Outline

Toward a Science of CE

The CTSA strategic goals include measuring the impact of CE on translational research through the rapidly evolving gold standard of health improvement at the population level.5 Consortium researchers have not consistently used terms and indicators necessary to advance a shared approach to assessing the contribution of CE and CEnR for clinical and translational research, improvement of population health outcomes, and alleviation of health inequities across diverse and often-underserved populations.15 CTSA assessment of CE started with a small set of activities, including consortium use of CE consultants to explore institutional-identified areas for CE improvement16; CE infrastructure development often involved constituting community advisory boards and reporting rudimentary counts of participants involved in CE activities (e.g., number of event attendees, pilot grant applications, researchers or community members educated). The expectation that translational research demonstrate population health benefits that are simultaneously cost-effective requires coordinated inquiry into community input within the research process and its contribution to accelerating the realization of pragmatic, or “real world,” value.

To address some of the challenges outlined above, we present a logic model to guide CE metrics (see Figure 1),17 an outgrowth of team science and the collaborative structure of the Outcomes Workgroup of the CTSA Community Engagement Key Functions Committee. In this article, we link the logic model to a typology of community–academic relationships to better evaluate the effect of CE on clinical translational science. The categories—engagement, collaboration, and shared leadership—focus attention on understanding how processes for establishing and maintaining long-term relationships with community-based groups build trust and improve research.18 The proposed typology is a way to minimize the inconsistent use of terms about community-engaged and participatory research in the current literature. Adopting clearly defined relationship terms will create a common framework for correlating and evaluating community–academic relationship types with specific CE and CEnR activities.

Back to Top | Article Outline

Model, Then Measure

Logic models have historically helped to articulate theories of change by describing how interventions are intended or assumed to produce outcomes; they are useful especially in new fields to facilitate hypothesis-driven research and evaluation. As a contributor to change theories, logic models can help research partners demonstrate and respond to epistemological challenges. Several recent logic models provide a structure to organize the study of community–academic research partnerships and their impacts.14,19–22 In particular, one ongoing NIH-funded project on community-based participatory research (CBPR) has identified existing instruments and tools for evaluating collaborative research partnerships.23 The project has put onto the Web a matrix linking variables to a CBPR conceptual model with questionnaire items that can be used to examine the correlations between partnering processes and specific practice, policy, and health outcomes within the model.24,25 A “key informant” survey for principal investigators, and a “community engagement survey” that consolidates relevant partnership and outcome questions into a comprehensive instrument, are also being tested.25,26 That project’s logic model and accompanying quantitative and qualitative instruments hold great potential for advancing an assessment across the consortium of CE and CEnR contributions to translational science.

Rather than a logic model, the British National Health Service commissioned a literature review of CE under the label public involvement in research. Conducted to increase the understanding of how public involvement influenced health sciences research, this meta-analysis concluded that public involvement added value throughout the research process by increasing recruitment for research of all types, improving clinical trial design, particularly in relation to the relevance for participants of outcome measures, and enhancing satisfaction for both researchers and research participants.27 This literature review also found limited evidence for public involvement in research due to the lack of structure and guidance from peer-reviewed journals for reporting on public involvement.27 The review revealed limitations in data collection and reporting on CE processes, which also remain largely unaddressed and underfunded within the CTSA consortium.

Historically, both institutional variation in valuing consortium participation and also the academic reward structure (i.e., advancement for individual efforts rather than collaborative ones) have impeded CE and CEnR assessment and evaluation.15,28 The challenges to cooperation across the consortium seem unacknowledged in the first request for applications (RFA) put forth under the auspices of NCATS, which encourages each CTSA program to “build on unique institutional strengths” while anticipating continuity in “areas that were highlighted in previous CTSA solicitations”; however, it is important to note that one previous area emphasized developing “innovative models for patient and community engagement in research.”29

The development of the CE logic model described in this article demonstrates a team science approach with the potential for advancing a collective agenda. The logic model encompasses a full infrastructure that can advance the study of CE and CEnR as characteristics of translational science at CTSA institutions by facilitating instrument development and subsequent collection of comparable data. The CTSA CE logic model “inputs” column shows common CE activities that were developed in response to early CTSA RFAs. In 2010, common CE activities across the consortium, which still remain today, include educating academic researchers about engaging communities in research and educating community members about clinical research, funding pilot programs to facilitate community–academic interaction, examining CEnR research milestones and timelines for tenure and promotion decisions, and developing administrative structures to support increased academic engagement with community groups, practice-based networks, and clinical and consortium partners.

On the basis of these activities, the logic model characterizes expected short-, intermediate-, and long-term outcomes as part of an overall infrastructure that supports published studies of community–academic participatory research. As a model of common CE activities at CTSA institutions, the designated “intermediate results” column (see Figure 1) provides a structure for assessment among consortium partners through shared measurement strategies such as institutional review board responsiveness to CEnR,13,30 measuring university capacity to support CEnR according to a measure of time from issuance of notice of award to issuance of subcontracts to community partners, the time needed to recruit the first participant, or perhaps the extent of racial and ethnic diversity among individuals actually enrolled in studies. More specific population health outcomes would depend on the specific investigations undertaken within each CTSA program and the development of measures (in collaboration with community partners) shared across the CTSA consortium to assess and to compare community-engaged health interventions.

Because the CE logic model names and focuses on CE activities and infrastructure outcomes, its usefulness extends beyond any single project with specific targeted outcomes and allocation of resources. This model provides an opportunity for understanding the contributions of CE and CEnR to translational science by integrating assessment focused primarily on clinical outcomes with assessments of how community–academic relationships influence the conduct of research and contribute to those clinical outcomes.

As a network of relationships, the CTSA consortium shares meaning-systems and organizes patterns of interaction, demonstrating the two primary characteristics of a community. Communities, like cultures, have been studied according to (1) cognitive processes and systems of meaning, and (2) structures and patterns of human social organization and interaction. These two methodological orientations to the study of community are incorporated into the recently revised Principles of Community Engagement,31 which explicates CE principles and how they function in contributing to improving public health. This second edition expands on methods for studying relationships that are inherently bidirectional, and advocates evaluation studies to contribute to a more complete understanding of CE. The logic model, therefore, provides the next step to supporting a collaborative evaluation structure of these codependent relationships and developing infrastructures.

Back to Top | Article Outline

The CE Logic Model: Structuring the Study of CE

In this article, we propose three categories of relationships, adapted from the CE continuum within the Principles of Community Engagement, to structure inquiry into and measurement of constructs within the logic model in order to learn how community participation may advance the translational science goal of linking clinical research to improved health in communities and the nation. The three relationship categories for structuring knowledge development—engagement, collaboration, and shared leadership—represent distinct, though not mutually exclusive, relationship types for the study of CTSA-sponsored CE activities.31

Engagement signifies an intention to exchange information and possibly resources (including money) through an individual event or a short-term series of events (including clinical trials). Outcome expectations for all participants are indeterminate because of the absence of ongoing information exchange, dialogue, and shared goals.

Collaboration signifies an intention for the members of a partnership to cooperate over time for the purpose of achieving specified goals; these goals may or may not be shared. Partners determine the duration of information and resource exchange and establish measurable goals, typically from a particular perspective and potentially shaped by formal agreements and defined responsibilities.

Shared leadership signifies an intention for the partnership to achieve shared goals. Partners share resources and information to enhance capacity for bidirectional exchange and achieve expected outcomes. Partners develop a shared participatory evaluation plan, supported through information exchange and dialogue, with assessment focused on activity implementation, on specific health and broader capacity outcomes, and on sustainability.

By studying relationships and the institutional variations in these relationship categories, it becomes possible to study CE and CEnR activities within unique local contexts and to collect comparable data of how academic engagement in community life influences clinical research methods, community health outcomes, and strategies to inform health policy. The collection of comparable data depends on consortium efforts to develop shared instruments for evaluating relationships developed through CE and CEnR.

The first step in the use of the logic model may be for each individual CTSA program to organize its vision of CE relationships using the above typology. No institution pursues every aspect of CE represented in the logic model, or specific CE activities, in precisely the same way. Each CTSA program could assess the variation in its multiple CE projects matched against the program’s vision, and then use the logic model to identify the potential outcomes on the basis of that assessment. The logic model’s concise summary of input activities could inform evaluation and could help to shift the study of CE and CEnR from specific projects and goals toward the types of relationships that support project development, protocol implementation, and shared interpretation and dissemination of results.16 However, the challenge for the consortium is not only to build infrastructure that connects researchers to communities but equally to build community infrastructure to support the implementation and dissemination of research findings in order to best foster health within each community context. As current experience and various Institute of Medicine reports make clear, knowledge and evidence are not in themselves sufficient to improve the quality of care.32,33

With the CE logic model providing a point of departure for CTSA programs, an additional challenge exists in marshaling cooperation and collaboration among consortium members to identify and deploy metrics and measurement strategies to study how community–academic relationships produced through CE and CEnR activities advance translational science. Existing evaluation methods and metrics—which are emerging among CBPR and other participatory evaluation studies to link participation processes with practice, system, policy, and health outcomes—can help advance this effort.22–24

Back to Top | Article Outline

Trust as Core to Productive Relationships

Although the logic model provides a set of inputs and short-term to longer-term outcomes, the question remains: How does change occur, and what processes correlate with or predict outcomes? The logic model organizes activities and expected outcomes while presuming that relationships, which may differ by vision and practice across the three types defined here, make a difference, especially those distinguished by synergy and trust. A recent realist evaluation review in the Milbank Quarterly applied partnership synergy as an intermediate theory of change across multiple partnerships, showing the value of relationships in contributing to outcomes.10,34

The concept of trust has been well articulated in the literature as a key CE metric, with assessment typically focused on community attitudes toward researchers.35 However, trust is a social phenomenon and therefore subject to analysis and interpretation from different perspectives.35,36 Dynamic and context-bound, trust is a mediator of cooperation; both trust and distrust shape relationships and are direct attributes of bidirectionality.16,34,37 Understanding trust may also enable researchers to avoid reifying or objectifying any particular community.38 Whereas the CE logic model situates bidirectional trust as a short-term result, all three relationship categories involve trust and are found throughout the logic model (e.g., between and within communities and research/science teams, organizations, and institutions). Trust as a measure of partnership strength, modification, and reevaluation requires a multimethods evaluation strategy.12,39–42

Many stories have been told about community distrust of research. The Tuskegee syphilis study involving African Americans,43–45 secret radiation experiments on mentally disabled children,46 the recent Havasupai lawsuit against Arizona State University for consent form violations,47 and “good will” studies that portray academic researchers as the sole possessors of research knowledge and skill, contribute to the history of community distrust of research.48 “Mosquito,” “helicopter,” and “seagull” research are among the negative characterizations of researchers who minimally interact with communities or impose their own context-bound views while extracting value for their own benefit, subsequently abandoning communities once their purposes are met.49–51 Stories of inequities in academic–community relationships produce feelings of disenfranchisement, marginalization, and disrespect for local experience and knowledge, particularly among vulnerable communities that traditionally lack a voice within the academy.52,53 A study of researcher attitudes about the minority community around one AHC found direct correlations between researchers’ discomfort at being in community settings and the extent to which those researchers engaged the community through research.54 Given the number of academic medical centers that are geographically proximate to minority neighborhoods and populations, other CTSA programs might benefit by replicating this study question.

Community–academic research partnerships that address public health problems and health disparities expand the national capacity to reduce or eliminate seemingly intractable health inequities.10,55–58 However, some researchers encounter communities that seek to limit partnership scope (e.g., solely to advise on a project or to broker subject/participant recruitment); such researchers contend that assuming communities always prefer a more engaged community-based participatory approach is presumptuous.59–61 Researchers also note that variability in engagement and partnership strength may result from the extent of trust and the approach to relationship consolidation or formalization,62 from strategies for disseminating findings and translating research into practice and policy,63 and from improved community assets, research capacity, and public health outcomes.64,65

The recognition of the importance of measuring trust within academic–community relationships has been growing, beginning with one of the earliest tools from the Prevention Research Centers.66 In addition, the University of Rochester developed a survey to evaluate community trust of its AHC as part of its CTSA evaluation. Finally, as part of its national CBPR cross-study, the University of New Mexico has developed trust measures, encompassing a spectrum from mistrust and proxy trust to ideal trust, for its Internet survey tool of community–academic partnerships as well as for its interview questions.22,35 The hypothesis is that partnerships with greater collaborative and shared leadership would have less mistrust and functional trust, and more evidence of ideal and sustainable trust.

Within the CTSA consortium, the construct of trust provides a collective opportunity for all sites to evaluate trust both as an independent and a dependent variable. Specific CEnR projects could use the measures for their own partnerships; yet, on a broader scale, each CTSA program could conduct data collection within its community advisory boards and academic leadership committees to assess trust development over time and across types of engagement. Such a study of trust could also enhance the study of each CTSA program’s capacity to function in multidisciplinary teams, the capability to adapt to complex situations, and a commitment to engagement and partnership that transcends specific projects.20,67

In sum, the hypothesis for engaging a community in research is that intentionally structured and respectful academic and community relationships can create shared meanings and structures to advance translational research and result in improvements in the health of communities and the nation.64 The CTSA consortium is moving toward developing pragmatic clinical and community trials that will inform this hypothesis only if the research outcomes data inform both the specific research question and the contextual relational (i.e., methodologic and implementation) components of the intervention. In essence, research cannot leave the controlled laboratory environment and then ignore its relationship to the social contexts.

Back to Top | Article Outline

Supporting the Full Spectrum of Translational Research

Translational research is often described as moving knowledge from bench to bedside or moving from an academic research focus on discovery into a clinical practice, primary care, and community setting focus on application.68,69 Translational science has also been accorded the lofty objective of facilitating knowledge transfer throughout the entirety of the U.S. health care system.70 A successful translational project requires both CE, with an emphasis on the bidirectionality of knowledge moving between the community and the academy, and relationships characterized by productive partnerships.

We have endeavored to show why CTSA programs, individually and collaboratively, should use the CE logic model to evaluate and assess CE activities according to three relationship categories: engagement, collaboration, and shared leadership. Further, we encourage consortium research teams to use trust and synergy as core metrics to organize assessment of their relationships. Without a coordinated evaluation strategy, the systematic evidence and insights necessary to assess how CE transforms the research process or how it advances the translational science goal of improving health will not be generated.

Recognizing individual variation between, and competing interests within, CTSA institutions, the NIH CTSA/NCATS Integration Working Group recommended that institutions participating in the consortium set aside and contribute internal funds and resources in predictable amounts to sustain CE and CEnR infrastructure. In response, translational researchers must commit to collaborating and to developing shared metrics to study CE and CEnR, and to determining whether viable and productive community–academic relationships support the full spectrum of translational research.28 Consortium participants can take the lead by committing to transparency and by collaboratively approaching CE and CEnR metrics and instrument development and assessment as an open-source project; the CTSA Consortium Coordinating Center provides infrastructure functionality to support open-source knowledge development.

With such cooperation within and between academic institutions and communities, we will be able to collectively and more effectively address inequities and redress disparities. Demonstrating engagement, collaboration, and shared leadership within the consortium will make it possible to realize the goal of translational research expressed in the long-term results part of the CE logic model: better science, better answers, better population health.

Back to Top | Article Outline

References

1. Eder M, Tobin JN, Proser M, Shin P. Special issue introduction: Building a stronger science of community-engaged research. Prog Community Health Partnersh. 2012;6:227–230
2. National Institutes of Health. . NIH launches national consortium to transform clinical research. NIH News. October 3, 2006 http://www.nih.gov/news/pr/oct2006/ncrr-03.htm. Accessed May 7, 2013
3. Coller BS, Califf RM. Traversing the valley of death: A guide to assessing prospects for translational success. Sci Transl Med. 2009;1:10cm9
4. Homer-Vanniasinkam S, Tsui J. The continuing challenges of translational research: Clinician–scientists’ perspective. Cardiol Res Pract. 2012:2012 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3459259/. Accessed May 9, 2013
5. . CTSA Clinical and Translational Science Awards. Strategic Goal Committee 4—Enhancing the Health of Our Communities and the Nation. https://www.ctsacentral.org/committee/sg4-enhancing-health-our-communities-and-nation. Accessed May 9, 2013
6. Califf RM, Berglund LPrincipal Investigators of National Institutes of Health Clinical and Translational Science Awards. . Linking scientific discovery and better health for the nation: The first three years of the NIH’s Clinical and Translational Science Awards. Acad Med. 2010;85:457–462
7. . Community–Campus Partnerships for Health Web site. http://www.ccph.info. Accessed May 7, 2013
8. Kellogg Health Scholars. . Community track. http://www.kellogghealthscholars.org/about/community.cfm. Accessed May 7, 2013
9. Carnegie Foundation for the Advancement of Teaching. . Classification description: Community engagement elective classification. http://classifications.carnegiefoundation.org/descriptions/community_engagement.php?key=1213. Accessed May 7, 2013
10. Stokols D, Fuqua J, Gress J, et al. Evaluating transdisciplinary science. Nicotine Tob Res. 2003;5(suppl 1):S21–S39
11. Frechtling J, Raue K, Michie J, Miyaoka A, Spiegelman M. The CTSA National Evaluation Final Report. https://www.ctsacentral.org/sites/default/files/files/CTSANationalEval_FinalReport_20120416.pdf. Accessed May 7, 2013
12. Mâsse LC, Moser RP, Stokols D, et al. Measuring collaboration and transdisciplinary integration in team science. Am J Prev Med. 2008;35(2 suppl):S151–S160
13. Trochim W, Kane C, Graham MJ, Pincus HA. Evaluating translational research: A process marker model. Clin Transl Sci. 2011;4:153–162
14. Hogle J, Hower CJ, Dunham N, et al. Evaluating community engagement in research in a clinical and translational science context. Definitions Interest Group of the CTSA Evaluation Key Function Committee (unpublished).
15. Selker HP, Califf RM. The need for academic leadership in full-spectrum translational research. Clin Transl Sci. 2011;4:78–79
16. Carter-Edwards L, Cook JL, McDonald MA, Weaver SM, Chukwuka K, Eder MM. Report on CTSA consortium use of the community engagement consulting service. Clin Transl Sci. 2013;6:34–39
17. Christoffel KK, Eder M, Kleinman LC, Hacker K Developing a logic model to guide community engagement metrics. May 13–14, 2010 Arlington, Va Poster presented at: Partnering to Improve Health: The Science of Community Engagement https://www.ctsacentral.org/sites/default/files/documents/LogicModel_May_2010_0.pdf. Accessed May 7, 2013
18. . Institutional Clinical and Translational Science Award (U54) Request for Applications (RFA) Number: RFA-RM-09-004. http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-09-004.html. Accessed May 7, 2013
19. National Institute of Environmental Health Sciences, U.S. Department of Health and Human Services. Partnerships for Environmental Public Health Evaluation Metrics Manual. NIH publication no. 12-7825. http://www.niehs.nih.gov/pephmetrics. Accessed May 7, 2013
20. Schulz A, Israel BA, Lantz P. Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships. Eval Program Plann. 2003;26:249–262
21. Center for the Advancement of Collaborative Strategies in Health. . The Web-based partnership self-assessment tool. http://www.partnershiptool.net/. Accessed May 7, 2013
22. Hicks S, Duran B, Wallerstein N, et al. Evaluating community-based participatory research to improve community-partnered science and community health. Prog Community Health Partnersh. 2012;6:289–299
23. Sandoval JA, Lucero J, Oetzel J, et al. Process and outcome constructs for evaluating community-based participatory research projects: A matrix of existing measures. Health Educ Res. 2012;27:680–690
24. . University of New Mexico School of Medicine. Interactive CBPR Conceptual Model. Product of “Research for Improved Health: A National Study of Community-University Partnerships.” http://hsc.unm.edu/SOM/fcm/cpr/cbprmodel.shtml. Accessed May 7, 2013
25. Pearson C, Duran B, Martin D, et al. CBPR Variable Matrix: Research for Improved Health in Academic–Community Partnerships. http://www.ces4health.info/find-products/view-product.aspx?code=FWYC2L2T. Accessed May 7, 2013
26. National Congress of American Indians Policy Research Center Study Documents. http://narch.ncaiprc.org. Accessed May 7, 2013
27. Staley K. Exploring impact: Public involvement in NHS, public health and social care research. http://www.invo.org.uk/posttypepublication/exploring-impact-public-involvement-in-nhs-public-health-and-social-care-research/. Accessed May 7, 2013
28. . NIH CTSA/NCATS Integration Working Group Recommendations. http://www.nih.gov/about/director/ncats/recommendations.pdf. Accessed January 26, 2013
29. . NIH/NCATS Institutional Clinical and Translational Science Award (U54) RFA. http://grants.nih.gov/grants/guide/rfa-files/RFA-TR-12-006.html. Accessed January 20, 2013
30. Drezner M, Cobb N. Efficiency of the IRB review process at CTSA-sites. https://www.ctsacentral.org/sites/default/files/documents/2_drezner.pdf. Accessed May 7, 2013
31. McCloskey DJ, Aguilar-Gaxiola S, Michener JL, et al. Principles of Community Engagement. 20112nd ed Bethesda, Md National Institutes of Health NIH publication no. 11-778
32. Nutting PA, Crabtree BF, Miller WL, Stewart EE, Stange KC, Jaén CR. Journey to the patient-centered medical home: A qualitative analysis of the experiences of practices in the National Demonstration Project. Ann Fam Med. 2010;8(suppl 1):S45–S56–S92
33. Institute of Medicine of the National Academies. Learning Health System in America Series. 2011 Washington, DC National Academies Press
34. Jagosh J, Macaulay AC, Pluye P, et al. Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. Milbank Q. 2012;90:311–346
35. Lucero JE, Wallerstein NTing-Toomey S, Oetzel J. Trust in community–academic research partnerships: Increasing the consciousness of conflict and trust development. Sage Handbook of Conflict Communication. 20132nd ed Thousand Oaks, Calif Sage Publications:537–563
36. Lucero J. Community engaged research: Measurement of process and outcomes. https://www.ctsacentral.org/documents/cbpr-interactive-model-presentation. Accessed May 7, 2013
37. Carnwell R, Carson ACarnwell R, Buchanan J. Understanding partnerships and collaboration. Effective Practice in Health and Social Care: A Partnership Approach. 2005 Maidenhead, England Open University Press:3–20
38. Berger PL, Luckmann T The Social Construction of Reality: A Treatise in the Sociology of Knowledge. 1966 Garden City, NY Doubleday
39. Stokols D, Hall KL, Taylor BK, Moser RP. The science of team science: Overview of the field and introduction to the supplement. Am J Prev Med. 2008;35(2 suppl):S77–S89
40. Meade CD, Calvo A. Developing community–academic partnerships to enhance breast health among rural and Hispanic migrant and seasonal farmworker women. Oncol Nurs Forum. 2001;28:1577–1584
41. Sapienza JN, Corbie-Smith G, Keim S, Fleischman AR. Community engagement in epidemiological research. Ambul Pediatr. 2007;7:247–252
42. Viswanathan M, Ammerman A, Eng E, et al. Community-Based Participatory Research: Assessing the Evidence. Evidence Report/Technology Assessment: Number 99. AHRQ publication 04-E022-2. http://archive.ahrq.gov/clinic/epcsums/cbprsum.htm. Accessed May 7, 2013
43. Jones J Bad Blood: The Tuskegee Syphilis Experiment. 1981 New York, NY Free Press
44. Corbie-Smith G, Thomas SB, Williams MV, Moody-Ayers S. Attitudes and beliefs of African Americans toward participation in medical research. J Gen Intern Med. 1999;14:537–546
45. Washington HA Medical Apartheid: The Dark History of Medical Experimentation on Black Americans From Colonial Times to the Present. 2006 New York, NY Doubleday
46. Welsome E The Plutonium Files: America’s Secret Medical Experiments in the Cold War. 1999 New York, NY Dial Press
47. . Havasupai tribe files $50 M lawsuit against Arizona State University. East Valley Tribune. March 17, 2004:1
48. Campbell-Voytal K. Phases of “pre-engagement” capacity building: Discovery, exploration, and trial alliance. Prog Community Health Partnersh. 2010;4:155–162
49. Gbadegesin S, Wendler D. Protecting communities in health research from exploitation. Bioethics. 2006;20:248–253
50. Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006;7:312–323
51. Cochran PA, Marshall CA, Garcia-Downing C, et al. Indigenous ways of knowing: Implications for participatory research and community. Am J Public Health. 2008;98:22–27
52. Shavers VL, Lynch CF, Burmeister LF. Racial differences in factors that influence the willingness to participate in medical research studies. Ann Epidemiol. 2002;12:248–256
53. Boulware LE, Cooper LA, Ratner LE, LaVeist TA, Powe NR. Race and trust in the health care system. Public Health Rep. 2003;118:358–365
54. Goldberg-Freeman C, Kass N, Gielen A, Tracey P, Bates-Hopkins B, Farfel M. Faculty beliefs, perceptions, and level of community involvement in their research: A survey at one urban academic institution. J Empir Res Hum Res Ethics. 2010;5:65–76
55. . Centers for Disease Control and Prevention. Prevention Research Centers. http://www.cdc.gov/prc/index.htm. Accessed May 7, 2013
56. Falk-Krzesinski HJ, Börner K, Contractor N, et al. Advancing the science of team science. Clin Transl Sci. 2010;3:263–266
57. Abrams DB. Applying transdisciplinary research strategies to understanding and eliminating health disparities. Health Educ Behav. 2006;33:515–531
58. Holmes JH, Lehman A, Hade E, et al. Challenges for multilevel health disparities research in a transdisciplinary environment. Am J Prev Med. 2008;35(2 suppl):S182–S192
59. Hood NE, Brewer T, Jackson R, Wewers ME. Survey of community engagement in NIH-funded research. Clin Transl Sci. 2010;3:19–22
60. Jones L, Wells K, Norris K, Meade B, Koegel P. The vision, valley, and victory of community engagement. Ethn Dis. 2009;19(autumn suppl 6):S6-3–S6-7
61. Jones L, Meade B, Forge N, et al. Begin your partnership: The process of engagement. Ethn Dis. 2009;19(autumn suppl 6):S6-8–S6-16
62. White-Cooper S, Dawkins NU, Kamin SL, Anderson LA. Community–institutional partnerships: Understanding trust among partners. Health Educ Behav. 2009;36:334–347
63. Minkler M, Vásquez VB, Warner JR, Steussey H, Facente S. Sowing the seeds for sustainable change: A community-based participatory research partnership for health promotion in Indiana, USA and its aftermath. Health Promot Int. 2006;21:293–300
64. Roussos ST, Fawcett SB. A review of collaborative partnerships as a strategy for improving community health. Annu Rev Public Health. 2000;21:369–402
65. Parker EA, Israel BA, Robins TG, et al. Evaluation of Community Action Against Asthma: A community health worker intervention to improve children’s asthma-related health by reducing household environmental triggers for asthma. Health Educ Behav. 2008;35:376–395
66. . Centers for Disease Control and Prevention. PRC Partnership Trust Tool. http://www.cdc.gov/prc/program-material/partnership-trust-tools.htm. Accessed May 7, 2013
67. Minkler M, Wallerstein N CBPR for Health: From Process to Outcomes. 20112nd ed San Francisco, Calif Jossey-Bass
68. Westfall JM, Mold J, Fagnan L. Practice-based research—“Blue highways” on the NIH roadmap. JAMA. 2007;297:403–406
69. Wolf SH. The meaning of translational research and why it matters. JAMA. 2008;299:211–213
70. Dougherty D, Conway PH. The “3T’s” road map to transform US health care: The “how” of high-quality care. JAMA. 2008;299:2319–2321
© 2013 by the Association of American Medical Colleges