Health disparities that lead to uneven access and quality and high costs will persist without a community-engaged research (CEnR) agenda that finds answers to both medical and public health questions. Concerned about the deficits in applying new research findings to the health problems our communities face—and about the reluctance of community members to participate in reserach—policy makers, funders, community leaders, and academic health centers (AHCs) are exploring how community engagement (CE) and CEnR can assist and enhance in their mission.1–7
The stakeholders most crucial to a shift toward CE and CEnR are those holding power in AHCs—the umbrella organizations that comprise a medical school, one or more related teaching hospitals, clinics, and sometimes other entities.8–10 A key challenge is to find ways to align the goals of medical school deans, hospital CEOs, and leading basic and clinical scientists with those of community stakeholders such as public health departments, neighborhood organizations, and elected officials, so that all begin working toward one goal: improving the health of local communities and the nation.
Perhaps the biggest barriers AHCs face in working toward a translational CEnR agenda are a result of AHC culture. The historical structures and processes of an AHC—including the complexities of how institutional review boards (IRBs) operate, accounting practices, and indirect funding policies, as well as tenure and promotion paths—can inhibit, discourage, and stifle partnerships outside of the AHC. Changing institutional culture requires support from the leadership and commitment of top decision makers.3,8,10,11 The Clinical Translational Science Award (CTSA) consortium has been working together to advance CEnR through a variety of channels. The CTSA consortium—launched in 2006 and consisting of 60 AHCs in 30 states and the District of Columbia—has as its primary goal to speed the process of translating basic research discovery into clinical application, clinical practice, and, ultimately, improved population health. The consortium is organized into 5 strategic goal committees (e.g., enhancing the health of our communities and the nation) and 14 key function committees (e.g., CE, comparative effectiveness research, education and career development, evaluation, informatics). The CE key function committee (CE-KFC) comprises over 200 members from each CTSA institution and various federal agencies. Its mission is to implement a successful broad plan of community and practice engagement among the CTSA sites by sharing knowledge, expertise, and resources. The goal of the CE-KFC is to effectively engage communities and practices in the translational research process via bidirectional dialogue. The main areas of focus include community and practice outreach, access, and dissemination of the translational research process via bidirectional dialogue.
The CE-FKC began by working closely with the National Institutes of Health (NIH) Director's Council of Public Representatives (COPR) to advance CEnR. The COPR is a federal advisory committee consisting of members of the public who advise the NIH director on issues related to public participation in NIH activities. NIH selects new COPR members every year, to serve an average of four-year terms. In 2010, COPR developed a framework that includes a set of values linked with strategies and expected outcomes. COPR defined CE in research as a process of inclusive participation that supports mutual respect of values, strategies, and actions for authentic partnership of people affiliated with or self-identified by geographic proximity, special interest, or similar situations to address issues affecting the well-being of the community of focus.12–15 COPR identified five core principles: definition and scope of CE in research; strong community–academic partnerships; equitable power and responsibility; capacity building; and effective dissemination of plans.12
Building on these principles, we are proposing five steps that AHCs can undertake to prepare their institutions for CE and CEnR. They are
- Define community and identify partners.
- Learn the etiquette of CE.
- Build a sustainable network of CEnR researchers.
- Recognize that CEnR will require the development of new methodologies.
- Improve translation and dissemination plans.
This is not a linear set of directives but, rather, a set of considerations to undertake concurrently.
Define Community and Identify Partners
How do we define community? There are multiple types and layers of potential community partners for AHCs to work with, including neighborhood organizations and leaders, faith-based organizations, community-based health practitioners, and governmental agencies. This diversity is a reflection that community is a fluid concept in which membership can be by choice, affiliation, common interest, or history; or by innate traits such as gender, race, and sexual orientation. People hold multiple memberships in communities, and researchers should view communities as complex systems composed of individuals, interest areas, experiences, and sectors,16–18 requiring preparation and often guidance for successful engagement. The following section provides examples of community research partnerships which may be relevant for AHCs, grouped according to local agencies, municipal infrastructure, health care service, and health care services research.
One of the most common partnerships is between AHCs and local community-based organizations. Examples include the partnerships between AHCs in Massachusetts and the Center for Community Health Education Research and Service, Inc., in Boston,18 or UCLA–Charles R. Drew University and Healthy African American Families in Los Angeles.14 AHCs also have formed strategic partnerships with patients and families with specific diseases to improve outcomes, as Cincinnati Children's Hospital did with families with children with cystic fibrosis to improve the continuous quality improvement process in delivering care.19 At University of California, Davis, the MIND Institute developed a new blood spot technique to inexpensively identify mutations in the fragile X gene and has closely collaborated with community partners in community settings so that appropriate treatment can be given.20
Every AHC should also consider being closely aligned with city councils as well as state and local public health departments. AHCs and public health departments both consider themselves as being the “front line” of health care, but they approach population health from different yet complementary lenses. Public health departments have access to epidemiological data and community health needs assessments as well the as existing partnerships and capacity necessary to affect large environmental and population changes, which can complement the more individual and disease approach traditional to AHCs. A good example of an academic/health department collaboration is the Durham Community Health Network, a partnership between Duke Health System, the Durham Public Health Department, and local primary care practices to improve care and health outcomes in Durham, North Carolina.21
Health care service
Safety net clinics—such as federally qualified health centers—are in an excellent position to address disparities in access and quality of care as they serve the vulnerable populations facing health care disparities. Too often, fiscal and bureaucratic barriers discourage AHCs from working with these clinics. At the same time, clinics are often wary of siphoning off limited funds to assist in research efforts. Successful partnerships include the University of New Mexico's Health Extension Rural Offices,22,23 Duke University Medical Centers division of community health and its “Just for Us” program,11,24 and Denver Health's relationships with safety net providers.25 Health Resources and Service Administration (HRSA)-funded health centers exist in most states and are increasingly interested in participating in research.
Health care research
Another crucial partner for AHCs are practice-based research networks (PBRNs). The majority of care delivered in this country is delivered in private physicians' offices; community, homeless, migrant, and school-based health centers; free clinics; and other settings outside of academic centers. Partnering with these professionals—doctors, nurses, physician assistants, dentists, social workers, medical interpreters—in crafting a research agenda and in disseminating results is crucial to translational medicine.26–29 Good examples of PBRN partnerships include the Michigan Clinical Research Collaboratory30 and the Oklahoma Physicians Resource/Research Network.31 The Centers for Disease Control's (CDC's) Prevention Research Centers—a network of academic, community, and public health partners that conducts applied public health research—offers another important opportunity for collaboration for AHCs.
Learn the Etiquette of CE
Before AHCs can build a CEnR agenda, they must first learn the principles of CE. There is often a long list of perceived benefits of CE to researchers, but for many underserved and historically abused communities, the benefits of medical research may be less clear. It is important for researchers to learn not only the principles of CE but also the underlying rationale for those principles.12
In 2011, the CDC and CE-KFC revised and updated the booklet “Principles of Community Engagement,” which offers up nine principles for planning, implementing, and succeeding in CE.32 In 2008, in conjunction with the Association for Prevention Teaching and Research (APTR), the CTSA-KFC hosted a series of regional workshops focused on CE. The discussions of these workshops were summarized in a monograph, “Researchers and Their Communities: The Challenge of Meaningful Community Engagement.”33 Some of the best practices in CE discussed at these workshops and in the monograph include
- Learning first about a community in terms of its history, culture, economic and social conditions, political and power structures, norms and values, demographic trends, and experience with research;
- Sharing power and showing respect;
- Including partners in all phases of research and planning; and
- Compensating community partners fairly.
Learn about a community in detail
Too often, researchers approach a community with a fully formed research proposal without understanding a community's needs and history. Worse, well-intentioned researchers themselves may not understand their own institution's history with a community.34,35 Many researchers and community partners reported that the best way to build this knowledge base of a community is through “unfunded connections” when there is not a specific project or grant in mind.33 Eliminating the restrictions inherent in having to answer to funding organizations can allow for more creativity and flexibility. For example, many researchers reported joining or forming local health coalitions and groups even though they weren't funded to do so. Forging relationships in these groups without a specific project or agenda in mind both builds trust and enables true partnership.33
Share power and show respect
Researchers need to be aware of power differentials and how communities conceptualize the varieties of power (level of education, length of residency, record of activism, etc.). The power differential creates challenges for developing meaningful and genuine partnerships. Both symbolically and practically, AHCs have access to resources (e.g., financial, institutional, people, social networks) that many communities (especially the underserved) do not. Open and respectful discussion of power issues can enable people to examine the power differentials and dynamics which exist in a community and in a partnership. This is a process that takes time, true commitment, and patience. Respect also means thinking of small, everyday concerns that might not occur to researchers but are important to community partners, such as holding meetings in settings where community members are familiar and feel comfortable and where it is easy for partners to park or are close to mass transit (as opposed to the often-complicated pay parking decks of AHCs); providing food and beverages at meetings; offering child care; and remembering to celebrate successes together.33
Include partners in all phases of research, beginning with planning
A key aspect of community-based participatory research that should apply to all forms of CEnR is planning research with community partners. Before writing research grants it is important to include community partners in defining the research agenda and in planning new projects. Planning processes which take time and patience that are unique to the CEnR could include mission statements with partners, goals of the project, discussions of roles, memorandums of understanding clarifying terminology, expectations and time lines, and focus groups with the underserved. One interesting example of community planning at every phase is the Toronto Teen Survey, a partnership between academic researchers, community partners, and a diverse cohort of teenagers living in Toronto.36
Compensate community partners fairly
Community partners—whether they are a physician group, a nonprofit neighborhood improvement organization, or a public health clinic—are often skeptical of partnerships with academic institutions that expect engagement without compensation. AHCs need to be flexible with policies and procedures and to collaboratively define fair compensation.33
Jones and Wells14 conceptualize the research activities of their partnership between Charles Drew Medical University and Healthy African American Families as having three implementation stages: “vision, valley and victory.” The “vision” is developing a shared view of the goal, the “valley” is doing the collaborative work and facing challenges, and the “victory” is completing the product and celebrating the process. Other ways to conceptualize community partnerships can be found in Table 1.
Build a Sustainable Network of CEnR Researchers
In addition to supporting researchers currently interested in CEnR, AHCs must also take an active role in building a sustainable network for training and cultivating future CEnR researchers. At many AHCs, community-engaged researchers are widely disbursed across different departments and schools; identifying and linking local experts is one of the early objectives of the CTSA program. Long-term, committed partnerships between communities and AHCs requires building a training pipeline that sparks interest in research in precollege students, makes CE a required competency for doctoral and medical students as well as residents and postdoctoral fellows, and rewards researchers for community-engaged projects that lead to improved outcomes. This can be done at a number of levels. For example, AHCs can provide training for all university students and researchers through required course work, self-paced modules, and certification. At Duke University Medical School, medical students, residents, and researchers wanting to work in the community go through a training process to ensure they will be effective and valued members of community projects.11 Some MD-MPH programs, such as the one offered by the University of North Carolina–Chapel Hill, offer good examples for building the pipeline.37
Another example is to offer community-based research tools and traineeships to develop the research capacity of medical and social service professionals who service underserved neighborhoods but have limited research experience. For instance, HRSA funds 53 Area Health Education Center (AHEC) programs and 221 affiliated AHEC centers in 45 states and the District of Columbia.38
Recognize That CEnR Will Require the Development of New Methodologies
One of the major obstacles to CEnR is a perception that randomized controlled trials (RCTs) are the single “gold standard” of biomedical research and that other study designs and methodologies are “second class” or inferior. Although RCTs are appropriate for many biomedical questions, the RCT's focus on rigorously controlling for complexity makes it a challenging method for multifactorial questions and dynamic situations, and limits the ability to generalize from the controlled clinical setting of a trial to the less controlled environment of community settings.39 The main population health issues facing the United States today—chronic disease, infectious disease, and injury prevention and control—and the individual and environmental contexts in which they occur are issues in which personal knowledge, skills and behavior, community and environmental factors, and policy choices interact to affect outcomes. Research that addresses these causes of morbidity and mortality but fails to incorporate the interactions of their social and environmental determinants in real life with real people will fail to translate into sustained improvements in human health.
Hawe and colleagues40 propose adapting RCTs to allow for complexity. The authors write: “Many people think that standardization and randomized controlled trials go hand in hand. Having an intervention look the same as possible in different places is thought to be paramount. But this may be why some community interventions have had weak effects.”40 In 2008, the British Medical Research Council recognized that standardizing the function and process of the interaction, rather than the process, is an alternative method of intervention integrity that allows local tailoring and can improve effectiveness. Alternative experimental methodologies include cluster randomization, stepped designs, and preference trials, in addition to quasi-experimental and observational studies.41 In addition to problems of complexity, RCTs also pose a variety of ethical challenges with many historically abused and clinically underserved communities who are hesitant to trust researchers. Potential healthy study participants and patients approached to participate in RCTs may interpret the word “trial” to mean that they are to take part in an “experiment,” in that the study has no safeguards and has never before been conducted on humans. One way to help overcome this perception is better outreach and targeted education about the necessity and positive aspects of clinical trials research.42–44 These outreach and educational efforts align well with the CTSA CE goal of enhancing the public trust in research.
West and colleagues39 propose additional alternatives to the RCT that empower participants. For example, instead of being randomly assigned to any one of available treatment or control arms after enrolling in “randomized encouragement designs,” participants are invited to participate in one particular treatment condition which is determined randomly before enrollment and described in full to the potential participant. Fully informed as to what one can expect, the individual then can decide whether or not to receive that treatment. Other approaches include “quantitative assignment designs,” which assign participants to treatment groups on the basis of a quantitative measure such as need, merit, or risk.39 Whereas group-randomized designs can also be very effective in community-level studies, Chatterji45 argues for extended–mixed method designs, which use qualitative and quantitative evaluation methods over the life span of a program or intervention.
Addressing a 2010 NIH conference, The Science of Dissemination and Implementation: Methods and Measurement, Rapkin46 advocated that communities of shared interest should form learning systems and should conduct successive studies that lead to refinement of key distinctions among interventions, types of populations, and settings. He also advocated for comprehensive dynamic trials that support learning systems by inventing and evolving interventions in place, drawing on multiple sources of information gained during the conduct of an intervention.
Improve Translation and Dissemination Plans
AHCs reward researchers for how often they publish in medical journals intended to communicate with other scientists. But most of these journal articles rarely trickle down to the level of nonscientists. Instead, researchers must work with their partners to creatively disseminate their findings to those who could benefit from them. To truly include community partners in building trust in science efforts as well as building a CEnR research agenda, translation and dissemination plans and strategies need to evolve beyond traditional medical journals. First, researchers need to initiate discussions about dissemination (and even routinely include dissemination plans in their research proposals) at the beginning of a project and target both scientific and lay publications. Next, grant budgets should reflect adequate time and funding to effectively manage the dissemination process. Academicians need to be willing to discuss—and, when appropriate, share—coauthorship with community partners. Also, having a dissemination work group, consisting of both key community and academic partners, will accelerate both lay and scientific communication. During the APTR-sponsored regional workshops on CE, researchers and community partners outlined these ideas further.
Partners advise thinking through financial, privacy, and ethical concerns of reporting results (or of failing to do so) at the beginning stages of a study. IRB considerations sometimes prohibit researchers from contacting research participants directly without their consent. Considering “opt-in” provisions in IRBs (“Is it okay to contact you with results?”) might encourage more participants to receive information about study findings. It is important to work with IRBs at the beginning stages of a project to address this concern.
Another challenge of CEnR is giving community members information in a timely and easy-to-understand manner so that they get practical knowledge out of the experience. Too often, waiting until the end of a study to report outcomes will be too late to keep participants engaged. It is important to find ways to keep partners updated throughout the process.
Community physicians are also too often overlooked at the end of studies. If they are not kept informed of study progress and results, they may be less motivated to participate in research, and less likely to engage in or refer patients for further studies. AHCs should not forget referring physicians in dissemination strategies.
Most important, it is crucial for AHCs to be creative in translating and disseminating findings and information to the community. From listservs and emails to “open mic” nights and town hall meetings, and partnering with local TV and radio stations (including ethnic media), AHCs should think of dissemination as a cyclical, recursive, and dynamic process that feeds the community information in formats that effect change and elicit new research questions. For many researchers, this is uncharted territory. Although insights from social marketing, journalism, public relations, and other disciplines can help, AHCs should evaluate their own best dissemination practices and share their results with each other.33
Challenges and Recommendations
Health disparities that lead to uneven access and quality and high costs will doubtless persist without a CEnR agenda that addresses both medical and public health questions. Some of the biggest barriers toward a national CEnR agenda, however, are the historical structures and processes of an AHC—including the complexities of how IRBs operate, accounting practices and indirect funding policies, and tenure and promotion pathways. Changing institutional culture starts with the leadership and commitment of top decision makers at an institution.8,11 Before engaging in the five steps we have proposed, it is important for stakeholders to examine the internal culture of their own AHC and decide whether and what kinds of changes are needed. This examination should include questions that are often not asked in academic settings. These questions may include: Is an institution a flexible and “community ready” partner? Are there tenure and promotion pathways for junior faculty interested in CEnR and community-engaged scholarship? Do IRBs include community members? Are members of IRBs trained in CEnR? Is CEnR a part of the mission/vision of the institution? Are there strategies in the AHC to institutionalize CEnR? Is there significant interest/expertise in the institutional leadership to make CEnR a key commitment of the medical institution?
Countless barriers to CEnR—small and large—exist throughout the culture and structure of AHCs. But these barriers and problems are not insurmountable (see Table 2 for strategies). For the U.S. health care system to be someday ranked the best in the world, leadership at every level of the health care system—from policy makers to community members—needs to envision safer, healthier communities, as well as new treatments that save lives. By aligning the motivations and goals of their researchers, clinicians, and community members into a vision of a healthier population, AHC leadership will not just improve their own institutions but also improve the health of the nation—one community at a time.
The authors wish to thank the NIH's Clinical and Translational Science Award consortium for their enthusiasm, support, and careful review. They would also like to thank their numerous community partners across the nation.
1. Northridge ME, Meyer IH, Dunn L. Overlooked and underserved in Harlem: A population-based survey of adults with asthma. Environ Health Perspect. 2002;110(suppl 2):217–220.
2. Nyden P. Academic incentives for faculty participation in community-based participatory research. J Gen Intern Med. 2003;18:576–585.
3. Ahmed SM, Beck B, Maurana CA, Newton G. Overcoming barriers to effective community-based participatory research in US medical schools. Educ Health (Abingdon). 2004;17:141–151.
4. Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream: Are researchers prepared? Circulation. 2009;119:2633–2642.
5. Leshner AI. Beyond the teachable moment. JAMA. 2007;298:1326–1328.
6. Viswanathan M, Ammerman A, Eng E, et al.. Community-Based Participatory Research: Assessing the Evidence. Summary, Evidence Report/Technology Assessment: Number 99. AHRQ Publication Number 04-E022-1. http://www.ahrq.gov/clinic/epcsums/cbprsum.htm
. Accessed November 2, 2011.
7. Rosenstock L, Hernandez L, Gebbie K, eds. Who Will Keep the Public Healthy? Educating Public Health Professionals for the 21st Century. Washington, DC: National Academies Press; 2003.
8. Dzau VJ, Ackerly DC, Sutton-Wallace P, et al.. The role of academic health science systems in the transformation of medicine. Lancet. 2009;375:949–953.
9. Lee TH. Turning doctors into leaders. Harv Bus Rev. 2010;88:50–58.
10. Brook RH. Medical leadership in an increasingly complex world. JAMA. 2010;304:465–466.
11. Michener JL, Yaggy S, Lyn M, et al.. Improving the health of the community: Duke's experience with community engagement. Acad Med. 2008;83:408–413.
12. Ahmed SM, Palermo AGS. Community engagement in research: Frameworks for education and peer review. Am J Public Health. 2010;100:1380–1387.
13. Fawcett SB, Paine-Andrews A, Francisco VT, et al.. Using empowerment theory in collaborative partnerships for community health and development. Am J Community Psychol. 1995;23:677–697.
14. Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. JAMA. 2007;297:407–410.
15. Minkler M, Wallerstein N. Community-Based Participatory Research for Health: From Process to Outcomes. 2nd ed. San Francisco, Calif: Jossey-Bass; 2008.
16. Minkler M. Community-based research partnerships: Challenges and opportunities. J Urban Health. 2005;82(2 suppl):ii3–ii12.
17. Israel BA, Eng E, Schulz AJ, Parker EA. Introduction to Methods in Community-Based Participatory Research for Health. San Francisco, Calif: Jossey-Bass; 2005.
18. Freeman E. Engaging a university: The CCHERS experience. Metrop Univ. 2000;11(2):20–27.
19. Gawande A. The bell curve. New Yorker. , 2004:82–91.
20. Hessl D, Berry-Kravis E, Cordeiro L, et al.. Prepulse inhibition in fragile X syndrome: Feasibility, reliability, and implications for treatment. Am J Med Genet B Neuropsychiatr Genet. 2009;150:545–553.
21. Cook J, Michener JL, Lyn M, Lobach D, Johnson F. Community collaboration to improve care and reduce health disparities. Health Aff (Millwood). 2010;29:956–958.
22. Kaufman A, Powell W, Alfero C, et al.. Health extension in New Mexico: An academic health center and the social determinants of disease. Ann Fam Med. 2010;8:73–81.
23. Kaufman A, Derksen D, Alfero C, et al.. The health commons and care of New Mexico's uninsured. Ann Fam Med. 2006;4(9 suppl):S22–S27.
24. Yaggy SD, Michener JL, Yaggy D, et al.. Just for Us: An academic medical center community partnership to maintain the health of a frail low-income senior population. Gerontologist. 2006;46:271–276.
25. Gabow P, Eisert S, Wright R. Denver Health: A model for the integration of a public hospital and community health centers. Ann Intern Med. 2003;138:143–149.
26. Fagnan LJ, Davis M, Deyo RA, Werner JJ, Stange KC. Linking practice-based research networks and clinical and translational science awards: New opportunities for community engagement by academic health centers. Acad Med. 2010;85:476–483.
27. Sloane PD, Dolor RJ, Halladay J. Increasing the role of practice networks in medical research. J Am Board Fam Med. 2009;22:348–351.
28. Westfall JM, Mold J, Fagnan L. Practice-based research—“Blue Highways” on the NIH roadmap. JAMA. 2007;297:403–406.
29. Mold JW, Peterson KA. Primary care practice-based research networks: Working at the interface between research and quality improvement. Ann Fam Med. 2005;3(1 suppl):S12–S20.
30. Schwenk TL, Green LA. The Michigan clinical research collaboratory: Following the NIH roadmap to the community. Ann Fam Med. 2006;4(1 suppl):S49–S54.
31. Mold JW, Aspy CA, Nagykaldi Z. Implementation of evidence-based preventive services delivery processes in primary care: An Oklahoma Physicians Resource/Research Network (OKPRN) study. J Am Board Fam Med. 2008;21:334.
32. CTSA Community Engagement Key Function Committee Task Force on the Principles of Community Engagement. Principles of Community Engagement. 2nd ed. Washington, DC: US Department of Health and Human Services; 2011. National Institutes of Health publication 11-7782. http://www.atsdr.cdc.gov/communityengagement/
. Accessed November 2, 2011.
34. Skloot R. The Immortal Life of Henrietta Lacks. New York, NY: Crown; 2010.
35. Lemkau J, Ahmed SM, Cauley K. The history of health in Dayton: A community–academic partnership. Am J Public Health. 2000;90:1216–1217.
36. Flicker S, Guta A, Larkin J, et al.. Survey design from the ground up: Collaboratively creating the Toronto teen survey. Health Promot Pract. 2010;1:112–122.
37. Harris R, Kinsinger LS, Tolleson-Rinehart S, Viera AJ, Dent G. The MD-MPH program at the University of North Carolina at Chapel Hill. Acad Med. 2008;83:371–377.
38. King J, Erin Fraher MPP, Ricketts TCI. History of AHEC and University Collaboration for Workforce Assessment. Natl AHEC Bull. 2007;23(2):15–17.
39. West SG, Duan N, Pequegnat W, et al.. Alternatives to the randomized controlled trial. Am J Public Health. 2008;98:1359–1366.
40. Hawe P, Shiell A, Riley T. Complex interventions: How “out of control” can a randomised controlled trial be? BMJ. 2004;328:1561–1563.
41. Craig P, Diepp P, Macintyre S. Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ. 2008;337:a1655.
42. Baum M. Clinical trials are ethically impossible. Lancet. 1993;341:812–813.
43. Saunders CM, Baum M, Houghton J. Consent, research and the doctor–patient relationship. In: Gillon R, ed. Principles of Health Care Ethics. New York, NY: Wiley; 1994.
44. Jenkins VFL. Reasons for accepting or declining to participate in randomized clinical trials for cancer therapy. Br J Cancer. 2000;82:1783–1788.
45. Chatterji M. Evidence on “what works”: An argument for extended-term mixed-method (ETMM) evaluation designs. Educ Res. 2005;34:14–24.
46. Rapkin B. New methods for participatory community-based intervention research: How can we develop a systematic understanding of change that matters? Partnering to improve health: The science of community engagement. Paper presented at: Association for Prevention Teaching and Research Conference; May 13–14, 2010; Arlington, Va.
This project has been funded in whole or in part with federal funds from the National Center for Research Resources, National Institutes of Health, through the Clinical and Translational Science Awards Program (CTSA), part of the Roadmap Initiative, Re-Engineering the Clinical Research Enterprise. The manuscript was approved by the CTSA consortium publications committee. (Grant numbers include RR024128-01, 05 KL2 RR024154-04, UL1RR025747, UL1 RR031973, and UL1RR024146.)