Launched in 2006, the Clinical and Translational Science Award (CTSA) Program constitutes a significant public investment estimated at $500 million in 2012.1 Currently, 61 academic health centers (AHCs) in 30 states and the District of Columbia participate in the CTSA Consortium.1 A central goal for the overall effort is to improve the health of local communities and the nation by “streamlining science, transforming training environments, and improving the conduct, quality, and dissemination of research.”2 With the goal of guiding and evaluating health interventions nationally and within specific communities, the CTSA institutions must collectively adopt an integrated set of community health indicators that reflect both public health* and community-driven priorities.3,4 (Here we define “community” broadly as any group defined by common geography [e.g., neighborhoods], membership [e.g., ethnicity], or experience [e.g., veterans].) Currently, many international and U.S. policy initiatives have created community health indicators.5–44 However, collec tively, these indicators pose—for those who would adopt them—multiple challenges including substantial overlap, ambiguity, and disagreement. Further, if the data collected do not reflect community priorities, indicators will ultimately lack relevance for the entire range of stakeholders and result in further divergence of metrics.
The purpose of this article is to provide an overview of the wide range of community health indicators and to propose the use of a systematic, common taxonomy for organizing and discussing them. We then illustrate the taxonomy’s application through a review of 21 health indicator projects. Finally, we discuss the intersection of the health needs of communities with the availability of data, plus the related, important need for striking a balance between the data requirements of AHCs and those of local public health departments and community partners.
Health Indicators: An Overview
The taxonomy of community health indicators we propose will help align CTSA activities and, in turn, allow us to define, measure, compare, and improve the effectiveness of interventions within and across CTSAs in a broad effort to improve the health of communities and, ultimately, the United States. At least four conditions are necessary for such a taxonomy. The taxonomy of community health indicators must:
- reflect community input, be relevant to communities, and have utility for both communities and researchers;
- be capable of identifying a set of measures or metrics that can be used to compare outcomes across multiple community health interventions so as to enable both comparative effectiveness research (CER; e.g., comparing different public health interventions3) and large-scale meta-analyses (e.g., aggregating results from similar community intervention studies for specific disease processes);
- achieve, or at least work toward, consensus on a shared language for community engagement processes, interventions, and health outcomes among community members, researchers, and public policy makers; and
- interface directly with the CTSA Community Engagement Logic Model.4
The Community Engagement Logic Model is a tool the CSTA Consortium has developed that focuses on building infrastructure to support relationships and collaboration between community and academic research partners. The logic model uses evidence-based structures and processes extant within CTSAs to support CTSA institutions’ engagement with community partners.4 The model includes inputs of community engagement activities and results in short-term outcomes (i.e., increased bidirectional trust and communication), intermediate or midterm outcomes (i.e., increased community capacity to engage in research or university capacity to engage with communities), and long-term outcomes (i.e., improved translation of science to new practices, policies, and programs that ultimately improve population health).
Why look at community health indicators?
The study of community health indicators is key to translating new scientific knowledge to applied health systems and practices consistently and broadly. It also allows investigators to conduct better CER, increases the ability to improve the relevance of public-health-related science, and makes it easier to competently measure underlying factors that impede adoption of new health findings.
Creating shared metrics among health researchers, government agencies, and communities themselves facilitates policy and decision making to improve population outcomes for a wide range of groups. By identifying indicators and best-fit metrics, researchers and communities are better able to more adequately define root causes and address complex issues related to health inequities.5
Finally, organizing health indicators into a community-engagement-focused taxonomy will allow organizations to leverage data collection for measures already employed for other purposes such as pay-for-performance, accreditation, or quality improvement programs. A unified taxonomy will advance community priorities while also improving the delivery of health care system services.3,4
Community health indicators and AHCs as data warehouses
Another reason for studying community health indicators is related to the current efforts to strengthen large health systems, including AHCs, that are vested in improving population health. The rapid advance of electronic health records, coupled with the development of large provider networks, puts many AHCs in the position of retaining detailed, primary data on the health status and health services utilization of many groups—sometimes most of the local population. The ability to build population health reports from primary health status and services data, and the challenges and limitations of this approach, are of key importance in the development of community health indicators.
Wide-scale deployment of electronic health records, in conjunction with traditional ongoing public health surveillance methods (e.g., cancer registries), can create a mix of real-time aggregated data, which can be supplemented by surveys targeted to particular communities. In addition, these new data sources permit real-time tracking of measures, and some of the resulting metrics—both biomedical (e.g., HgbA1C) and those related to health systems utilization (e.g., transportation for medical appointments, obesity prevention services, access to social support services, group counseling sessions)6—may be of considerable interest to communities. Further, members of the research community can often access these new data sources (e.g., electronic health records, targeted surveys), such that the sources serve as another bridge between the community and academia.
Key Literature Informing a Taxonomy of Community Health Indicators
As we began to develop our taxonomy, we reviewed relevant literature, including historical and government-related uses of health indicators.
Over several decades, various U.S. and international experts have identified their own lists of key health indicators in endeavors to focus health promotion and disease prevention efforts while monitoring changes in outcomes.7–10 The earliest community health indicator was the rate of infant mortality collected from the mid-19th through most of the 20th century8; and around 1910, the Russell Sage Foundation advanced the use of indicators by developing local surveys to measure factors affecting community health.8 More recently, in the 1980s, the World Health Organization convened the Berne Workshop in Switzerland to assess health promotion indicators.9
In 1990, the Institute of Medicine (IOM) established a committee to evaluate factors impacting health and to develop a set of health indicators specifically for the United States.7 This use of indicators continued through the 1990s, as interests turned to determining a community’s well-being as a mechanism to improve the planning, monitoring, and evaluation of community health initiatives.8 In 2000, the U.S. Department of Health and Human Services published Healthy People 2010, which outlined the first national plan for improving the health of the U.S. population; this document focused measurement on key population and individual health indicators. Since then, the IOM has synthesized its previous efforts and collaborated with the U.S. Department of Health and Human Service’s Community Health Indicators Project and County Health Rankings, setting the stage for the current decade with Leading Health Indicators—Healthy People 2020.7,10–12
Other organizations have also begun to study health indicators—and how they may relate to other measures. In 2005, the Community Indicators Consortium, sponsored by the Alfred P. Sloan Foundation, discussed differences and linkages between civil/community indicators and government performance measures.10 Some characteristics of the civil or community indicators that overlap with those of community health indicators are as follows: (1) both types of indicators are societally driven or socially determined (rather than focused on the performance of a service); (2) both address community conditions such as access to housing and health care services, the physical environment, and economic conditions (rather than the number and cost of services delivered); (3) both engage a wide range of stakeholders who are influenced through consensus building; and (4) both are usually implemented through grassroots, community-based, nongovernment organizations.10
Further, data related to health indicators are available through national public health agencies, such as the Centers for Disease Control and Prevention, which have been monitoring the health of the public for decades through a variety of surveillance tools including population-based surveys.13,14 Similarly, the Agency for Health Research and Quality compiles and reports national quality and disparity data, providing an annual assessment of clinical health outcomes.15,16 The data used for community health indicator development are thus becoming increasingly available as government and private foundations disseminate information to the public for practical application.
Finally, health indicators guide policy development and policy implementation. The U.S. Patient Protection and Affordable Care Act (ACA) mandates development of a core set of health indicators that will be used to evaluate the effectiveness of health care systems and interventions at the population level. The ACA combines a significant focus on individual health outcomes with a consideration of population-level health outcomes such as disease prevention, upgrading public health systems, and increasing access to clinical preventive services. In addition, the ACA’s National Prevention Strategy addresses issues related to the social determinants of health by encouraging Americans to live healthier lives through four aims: (1) building healthy and safe community environments, (2) expanding quality preventive services in both clinical and community settings, (3) empowering people to make healthy choices, and (4) eliminating health disparities.8
European Community Health Indicators
The taxonomy of standardized health indicators we propose is not without precedence. The European Community Health Indicators (ECHI) Project advanced the following four categories to serve as the conceptual basis for refining community health indicators across Europe: (1) demographic and socioeconomic factors, (2) health status, (3) determinants of health, and (4) health systems.9 These main categories have commonly been referred to as a basis for defining more specific health indicators.9,17–19 The ECHI Project indicators were developed to generate national and regional public health reports to shape policy; to create a logical framework for longitudinally monitoring health programs; to identify data gaps for prioritizing data collection and harmonization processes (i.e., step-by-step procedures used to arrive at a set of decisions); and to enable the establishment of a data-sharing infrastructure in the European Union.9 The EU consortium’s model focuses on nonmedical ecological determinants of health—those that emphasize mental health and social-cultural-environmental structures and processes.18
There are parallels between ECHI goals and several CTSA initiatives. First, ECHI goals and strategies echo those of CTSA Strategic Goal Four: “enhancing the health of our communities and the nation.”3 Like the CTSA initiative, ECHI represents a large-scale strategy across diverse populations that supports the improvement and achievement of equity in access, quality, and health care delivery. It is focused on a centralized information exchange to facilitate comparisons, disseminate best practices, and achieve health equity.3 The centralized European health care system model and the ECHI infrastructure are accelerating development of a common platform to disseminate health research findings and technology to community users, which will reduce barriers to communication and collaboration, strengthen public health relationships, increase community research capacity, and accelerate policy change.9
Despite many similarities, the ECHI and CTSA programs have differences, especially in the challenges each program faces. One challenge for ECHI is moving forward with the adoption of the common classification system. Another involves standardizing all health care systems such that each has similar basic care infrastructure components. In contrast, the U.S. system’s challenge is to centralize the health care system so that improvements can be rapidly disseminated and uniformly implemented across communities.
Over time, the differences between and lessons learned across these two major systems (EU and U.S.) will be mutually beneficial and informative.
Towards a Taxonomy of Community Health Indicators
As the literature shows, policy makers and leaders at various levels have established systems for monitoring change in health through health indicators. Fundamental is the need to provide common, high-quality, reliable, objective data that measure population health in areas where progress can be tracked over time.11,20 A set of indicators will support the evaluation of community engagement activities outlined in the CTSA Community Engagement Logic Model.4 Finding common ground between these CTSA community engagement measures and other data-driven evaluations will help achieve some economies of scale through the use of current metrics. Common metrics may also advance the development of electronic information systems and databases that can support community engagement evaluations and grant development. Most important, the development of a detailed taxonomy will serve as a guide for the production of public health reports and foster the dissemination and, when appropriate, the implementation of health research findings to communities.
A taxonomy is a particular classification system arranged in a hierarchical structure providing supra- and subtype relationships.21 Our research uncovered common concepts across the literature that, when compiled, fell into three ordered and nested categories. The community health indicator taxonomy provides a conceptual foundation for the 21 indicator projects we explore in this article. Synthesizing the use of indicators and measurement terms in these projects points to a simple hierarchy that can be expressed in a single sentence: Determinants of health have categories of community health indicators that include specific quantifiable measurements (see Figure 1). The hierarchy for our taxonomy is based on the need to express observations ranging from broad-based determinants of health to highly specific, quantifiable and measureable phenomena.
Determinants of health, at the topmost level, include the social, economic, and physical environment, as well as a person’s individual characteristics and behaviors.8,17–19 These determinants also include factors that combine to affect individual and community health, both directly and indirectly.8,20 Our review of indicator projects reveals a list of indicator types or classes that are organized within their respective, overarching determinants of health categories.
In the middle, community health indicators, more specific than determinants of health, but less specific than quantifiable measurements, are particular characteristics of an individual, population, or environment that can be measured and used to describe the health of that individual, population, or environment.12 Health indicators are considered to be tools21 with enough information or data to describe and compare (across individuals, populations, or environments) health statuses and health services.17,22,23
Quantifiable measures, at the lowest point in our hierarchy, are the standard reference points through which other points of information can be evaluated.24 Data measures can originate from various indicator categories including epidemiological, socioeconomic, geographic, health care utilization,25 health care quality,26 social capital,27 and resource distribution.28
We acknowledge that others have used these same terms for other purposes and to have other meanings; for example, some uses of the term indicator in programs such as the Baldridge Criteria for Performance Excellence in Health describe comparisons of processes or meta-comparisons, whereas measure indicates a more direct, data-driven evaluation.23 We hope that one benefit of the taxonomy we propose will be to standardize vocabulary.
Appendix 1 shows our taxonomy and indicates the major categories of indicators in use today within the organizing principle determinants of health. The content and user interface of the U.S. Department of Health and Human Services Health Indicators Warehouse (HIW), which catalogs a wide array of health indicators and classifies them into nested and searchable categories,12 provides an excellent example of a similar organizing principle. To illustrate, the Personal Behavioral Factors, a broad category of indicators in our taxonomy (see Appendix 1), is comparable to the HIW’s indicator descriptors for Health Behaviors and Early Childhood Experiences.
Summary of Community Health Indicator Projects Using the Proposed Taxonomy Approach
To develop a picture of current health indicator efforts, to identify gaps in current understanding and levels of consensus, and to guide future work, we present here a summary of indicator projects based on a review of available literature. Using our taxonomy (Appendix 1) and our hierarchy described above (Figure 1), we identified commonly used broad categories of determinants of health, extensive lists of categories of indicators, and many specific quantifiable measures. We organized health indicator projects into one of three jurisdictions: international (multi-country),20,29–32 national (country),7,11,12,24,26,33–35 or local (state, county, and/or special populations).17,36–41,45
Next, we ordered the types of health indicator projects by frequency at two levels. At one level, we ordered the broad Determinants of Health categories by the relative number of specific quantifiable measures identified for each in the literature; in other words, “Health System Services” had the highest number of specific quantifiable measures, while the more challenging to collect, but critically important “Social Structure” category had the fewest. At the second level, we ranked, within each Determinant of Health category, the indicators with the greatest representation across the various indicator projects; for example, within the Health System Services category, “access to health care provider” measures are discussed in 18 out of 21 projects, whereas “composite” measures are discussed in only one of these projects.
Finally, the shaded boxes in the Measures area of Appendix 1 denote at least one specific measure that has been identified within an indicator category.
Among the 21 indicator projects (many of them providing national and regional data) that we reviewed, we observed great variation among the specific quantifiable measures communities have used to describe and track health outcomes. Suggested indicators that lack a clear set of population-based, public health surveillance data measurements include those related to the role of social status and social capital,27,37 discrimination and stress,8 and perceptions of role in society (see Disparities, Social Cohesion, and Social Structure in Appendix 1). In 2003, the Task Force on Community Preventive Services provided an extensive list of variables which may measure a more local social environment’s impact on health36 rather than directly addressing either community well-being on a larger scale or even the interconnectivity and support among persons,43 community-based organizations, government,44 local businesses,46,47 and other built environmental conditions and resources.48 However, as illustrated by our taxonomy, more work needs to be done in this arena.
Working With Community Partners to Identify Local Health Indicators
Categories of indicators can become important guides for individuals and communities developing and planning health improvement interventions. However, these indicators clearly need to be streamlined and made relevant to individual communities. The current number of unconnected sets of various indicators are indicative of disagreement among community members and organizations serving communities. And, although some existing sets are reliable and valid, the sheer variety of these sets, across federal and other systems, creates enormous barriers to reaching consensus and often leads to parallel or overlapping indicator projects. Also, trust and confidence in both private and federal systems and in public health often do not exist among all consumers or in many underserved populations.48 Therefore, to promote community trust and to stimulate interest in the indicators, a next logical step is to involve community stakeholders, as well as health care professionals, researchers, and policy makers, in assessing existing indicator options and in identifying their own priorities.
The role of community partners
Over the last two decades, a growing number of communities, led by local health departments, clinical care systems, not-for-profit organizations, and local/county governments, have developed their own community health indicators relevant to their local or state context; one example is the Wisconsin County Health Rankings.45 Tasked with conducting community needs assessments as one of the three core public health functions, both state and local health departments currently prioritize health indicators, monitor health status, and investigate health problems in their community. These organizations regularly make their data available (e.g., via community health profiles, vital statistics, and health status) to constituents who, in turn, use the data to depict the health challenges and strengths of smaller geographic areas.45 These local health departments and other groups could come together to integrate indicators from surveillance systems, real-time electronic health records, and local data priorities (gleaned, for example, from targeted health surveys or environmental monitoring). Community stakeholders may especially welcome maps generated by Geographic Information System initiatives or visual portrayals of information to better use data as a positive force for strategic planning and health improvement.
Showing the impact of community engagement on health through evaluating community–academic partnerships requires evaluating both the process of partnering and the resulting impact of the partnership on the system (e.g., greater capacity, community empowerment, new policies, or clinical practice changes). Community involvement in creating logic models for change enables communities to document benchmarks for progress and to formulate hypotheses about which indicators and which partnership practices may enhance capacity or improve system change measures.49 This process of hypothesis testing and partnership consolidation can, in turn, contribute to population health changes.50
Engaging community groups also helps the community identify the health indicators that have the greatest potential to improve local well-being. Additionally, local neighborhoods may assist in conducting assessments of community needs and community strengths or assets so as to provide data for health improvement efforts, identifying areas of strength and leveraging these strengths to address needs or concerns.28
Some of the community agencies that have conducted their own needs assessments range from social service agencies to faith-based organizations, to hospitals, local funders, and community coalitions. Community groups use a variety of data sources and organizing principles. Local coalitions may obtain independent funding to assess health needs and establish indicators to monitor progress over time. They may use data supplied by state and local authorities and/or collect data on their own using community-organizing principles (e.g., trust building) to develop consensus on indicators. Many organizations demonstrate a tremendous ability to connect with their constituents, many of whom may be underserved or marginalized. A number of strategies for identifying community needs and assets, including Mobilizing for Action through Planning and Partnerships,51 Protocol for Assessing Community Excellence in Environmental Health,52 and Assessment Protocol for Excellence in Public Health,53 have been popularized. Although some universities may be involved as leaders in or organizers of these activities, it would be an advance for academic institutions to work more closely with communities to improve health at this bidirectional level.
The role of the CTSAs
Today, the CTSAs have a unique oppor tunity to contribute both to bridging the gap between academe and public/community health improvement and to documenting that change through an integrated set of health indicators. Our recommendation would be for CTSA institutions to include in their investigations and research protocols indicators representing each of the major Determinants of Health categories; that is, to examine factors related, for example, to health system services and general health status, to personal behavioral and community socioeconomic composition, and to social cohesion and social structure. As CTSAs engage with communities in selecting and measuring indicators, community stakeholders can participate in and add to already-extant community indicator projects.54,55 Local efforts to measure indicators, many of which already involve partnerships with universities, would make valuable contributions to the efforts to establish a standard set of indicators for the nation. CTSAs can therefore build on local connections and collaborations. CTSA institutions may provide specialized infrastructure and offer technical assistance, expertise, and resources while both honoring the work that has emerged from within community institutions and addressing areas that are of high priority to community members and leaders.54,55
Choosing the right combination
Our hope for this community health indicator development work is that it will allow individual communities to use indicators more meaningfully. A magnitude of data is available, especially as health systems such as AHCs deploy electronic health records, so determining which of the multiple basic community health indicators are appropriate for a specific community is important. The taxonomy allows leaders at the county or city level to review the indicators and measures available and to select those that are most appropriate for their purposes and constituents. Every community interested in improving the health of local populations must identify a health issue and target population and collect baseline data. For CTSA institutions and other universities, the greatest potential for improving the health of the community and nation will come through partnerships that blend the expertise and resources of universities with established community entities, including state and local departments of public health, as well as community-based organizations and grassroots groups.
Benefits to the taxonomy itself
In addition to standardized categories or indicators that could be adopted nationally, the CTSA Consortium has, as mentioned, developed an infrastructure logic model of community engagement structures and processes within CTSAs.4 This logic model posits short-term outcomes, intermediate-term outcomes (increases in community capacity to engage in research and university capacity to engage with communities), and long-term outcomes (i.e., improved translation of science to health practices, policies, and programs that, ultimately, improve population health). Though the logic model takes into account the congruence of community and academic interests and outlines community-based strategies for determining health indicators and desired outcomes, there is still a paucity of specific, quantifiable measures or metrics for systems-capacity or population-health changes. Many of the midterm capacity and long-term, system-wide outcomes that result from CTSA–community partnerships could themselves serve as indicators or benchmarks of progress towards population health changes.4
Seizing the Opportunity to Be Relevant to Communities
The IOM, after evaluating progress of the CTSAs, identified community engagement as one of “three crosscutting domains that … are integral to effectively advancing clinical and translational science.”1 In the same report, the IOM provided recommendations to strengthen the support of community engagement efforts and noted that community support “is critical in all phases of clinical and translational research from basic research to clinical practice and community and public health.”1 The CTSAs have a unique and timely opportunity through their community engagement programs and activities both to enhance academic–public–community partnerships and, through these partnerships, to support efforts to determine community health indicators at the national, state, and local levels. Armed with awareness of local community health activities, CTSA institutions are poised to be active players in health improvement efforts locally; they can provide infrastructure support, technical assistance, and leadership to the community health indicator development process. The challenge CTSA Consortium members have undertaken is to reconcile the potentially contradictory goals of, on one hand, arriving at standardized health indicators that allow for the monitoring and comparing of progress in improving public health outcomes and, on the other, respect for the interests of communities who want to retain ownership of the process for identifying health indicators that reflect local priorities rather than those imposed from outside. Minimally, the CTSA institutions and consortia need a shared language and a standardized hierarchy of categories of community health indicators. Possibly, they could also develop specific metrics for a core set of community health indicators, quantifiable outcome measures, and monitoring systems that not only allow for local, community organizations to begin to gather data more readily but also inform policy makers and the public on progress made in improving health.26
The taxonomy we propose here is explicitly designed to serve the needs of the CTSAs and communities throughout the nation. Further, this effort is designed to situate community health indicators in the CTSA context of translational science, enhance the methodological rigor of community-engaged research, and ultimately improve population health.
The authors wish to acknowledge and thank the following individuals who contributed to the development of this manuscript: Lawrence C. Kleinman, MD, MPH (Mount Sinai School of Medicine); Susan Lyons, JD (University of Alabama Birmingham); Carlar Blackman (University of North Carolina at Chapel Hill); and Rosalind Kirk, PhD (Michigan State University). They would also like to express appreciation to all the Clinical and Translational Science Award (CTSA) Community Engagement Key Function Committee Outcomes Workgroup participants for contributing to the fruitful exchange of ideas. The authors extend a special thanks to Leslie Boone, MPH, program manager at the CTSA Consortium Coordinating Center; to Donna Jo McCloskey, RN, PhD, G. Iris Obrams, MD, MPH, PhD, Andrea Sawczuk, DDS, PhD, MA, and Meryl Sufian, PhD, of the National Institutes of Health (NIH); and to the Booz Allen Hamilton Project Managers (Jenna Goldstein, Allison Heiser, Valerie Kokai, Nicole Morales) for their help and excellent assistance throughout the development of this article. The authors wish to also thank the NIH’s CTSA Consortium for their support and careful review of the manuscript. They would especially like to thank their numerous community partners across the nation.
* Distinguishing between public health and population health (both terms are used in this article) is important:• Public health refers to the “critical functions of state and local public health departments such as preventing epidemics, containing environmental hazards, and encouraging healthy behaviors.” (Kindig D, Stoddart G. What is public health? Am J Public Health. 2003;93:380–383.)• Population health refers to “the health outcomes of a group of individuals, including the distribution of such outcomes within the group.” (University of Wisconsin, Population Health Sciences. Improving Population Health: Policy, Practice and Research. What Is the Difference Between Population Health and Public Health? http://www.improvingpopulationhealth.org/blog/what-is-the-difference-between-population-health-and-public-health.html. Accessed January 7, 2014.)
4. Eder MM, Carter-Edwards L, Hurd TC, Rumala BB, Wallerstein N. A logic model for community engagement within the CTSA consortium: Can we measure what we model? Acad Med. 2013;88:1430–1436
5. Braveman P, Egerter S, Williams DR. The social determinants of health: Coming of age. Annu Rev Public Health. 2011;32:381–398
6. Hochman ME, Chen AY, Serota M. A view from the safety net. JAMA. 2013;309:2221–2222
7. Committee on the State of the USA Health Indicators; Institute of Medicine. . State of the USA Health Indicators: Letter Report. 2009 Washington, DC National Academies Press
8. Skocpol T, Jacobs LR Reaching for a New Deal: Ambitious Governance, Economic Meltdown, and Polarized Politics in Obama’s First Two Years. 2011 New York, NY Russell Sage Foundation
9. Kramers PGN. The ECHI project: Health indicators for the European community. Eur J Public Health. 2003;13(suppl 3):101–106
12. U.S. Department of Health and Human Services. . Health Indicators Warehouse. 2012 http://healthindicators.gov
. Accessed January 4, 2014
17. Bardehle D. Minimum health indicator set for South Eastern Europe. Croat Med J. 2002;43:170–173
18. Bauer G, Davies JK, Pelikan J. The EUPHID health development model for the classification of public health indicators. Health Promot Int. 2006;21:153–159
19. Fraser ED, Dougill AJ, Mabee WE, Reed M, McAlpine P. Bottom up and top down: Analysis of participatory processes for sustainability indicator identification as a pathway to community empowerment and sustainable environmental management. J Environ Manage. 2006;78:114–127
21. Salvador-Carulla L, Salinas-Pérez JA, Martín M, et al.GClin-SEP group. A preliminary taxonomy and a standard knowledge base for mental-health system indicators in Spain. Int J Ment Health Syst. 2010;4:29
26. Arah OA, Westert GP, Hurst J, Klazinga NS. A conceptual framework for the OECD Health Care Quality Indicators Project. Int J Qual Health Care. 2006;18(suppl 1):5–13
27. Macinko J, Starfield B. The utility of social capital in research on health determinants. Milbank Q. 2001;79:387–427, IV
28. Minkler M. Linking science and policy through community-based participatory research to study and address health disparities. Am J Public Health. 2010;100(suppl 1):S81–S87
32. Korkeila JA, Kovess V, Dalgard O-S, Madianos M, Salize H-J, Lehtinen V. Piloting mental health indicators for Europe. J Ment Health. 2007;16:401–413
36. Anderson LM, Scrimshaw SC, Fullilove MT, Fielding JETask Force on Community Preventive Services. . The Community Guide’s model for linking the social environment to health. Am J Prev Med. 2003;24(3 suppl):12–20
38. Watkins K, Horvitz-Lennon M, Caldarone LB, et al. Developing medical record-based performance indicators to measure the quality of mental healthcare. J Healthc Qual. 2011;33:49–66
39. Pincus HA, Spaeth-Rublee B, Watkins KE. The case for measuring quality in mental health and substance abuse care. Health Aff (Millwood). 2011;30:730–736
40. Ramos TB. Development of regional sustainability indicators and the role of academia in the process: The Portuguese practice. J Cleaner Prod. 2009;17:1101–1115
41. Kelly SM, O’Grady KE, Schwartz RP, Peterson JA, Wilson ME, Brown BS. The relationship of social support to treatment entry and engagement: The Community Assessment Inventory. Subst Abus. 2010;31:43–52
42. Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006;7:312–323
43. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Serv Res. 2007;42:1758–1772
44. Van Assche J, Block T, Reynaert H. Can community indicators live up to their expectations? The case of the Flemish City Monitor for livable and sustainable urban development. Appl Res Qual Life. 2010;5:341–352
46. Veleva V, Ellenbecker M. Indicators of sustainable production: A new tool for promoting business sustainability. New Solut. 2001;11:41–62
48. Cottler LB, McCloskey DJ, Aguilar-Gaxiola S, et al. Geographic and racial/ethnic differences in community needs, concerns and perceptions about health research: Findings from the CTSA Sentinel Network [published online February 14, 2013]. Am J Public Health.
49. Hicks S, Duran B, Wallerstein N, et al. Evaluating community-based participatory research to improve community-partnered science and community health. Prog Community Health Partnersh. 2012;6:289–299
50. Schulz AJ, Williams DR, Israel BA, Lempert LB. Racial and spatial relations as fundamental determinants of health in Detroit. Milbank Q. 2002;80:677–707, iv
54. Kulynych J, Heffernan KG. Community research partnerships: Underappreciated challenges, unrealized opportunities. JAMA. 2013;309:555–556
55. Davis MM, Clark SJ, Butchart AT, Singer DC, Shanley TP, Gipson DS. Public participation in, and awareness about, medical research opportunities in the era of clinical and translational research. Clin Transl Sci. 2013;6:88–93