Over the last decade, emergency preparedness communication campaigns and other interventions have encouraged individual and family-level preparedness, with messages emphasizing the benefits of household emergency preparedness kits. These messages have helped raise some public awareness about disaster readiness but, overall, have not resulted in significant behavior change.1 In addition, as the number and costs associated with disaster increase, communities, and not simply individual households, are challenged with making decisions about where to rebuild, how to recover, and how to address readiness for the next disaster. This recognition that whole community must convene to develop integrated disaster plans and prepare constituents for the magnitude and length of response and recovery is evidenced in national planning documents that call for community resilience and a heightened emphasis on community-level capacities and capabilities for disasters.2 For example, the National Health Security Strategy asks communities to consider the extent to which they have strong partnerships across government and nongovernmental organizations.3 Other frameworks such as the National Disaster Recovery Framework or Centers for Disease Control and Prevention's (CDC's) Public Health Emergency Preparedness Capabilities argue for community capacity building, yet in all of these policy documents, there is little information on the metrics that should or could be used in monitoring and evaluating community capacity or readiness.4,5
Disaster research suggests that there are several domains that capture aspects of community-level preparedness and serve as indicators of community capacity.6 The ability of the community to collectively marshal resources without external intervention is one component (ie, self-sufficiency). Another is the community's ability to engage diverse stakeholder groups in meaningful ways across the disaster continuum (ie, strong partnerships). Finally, recent disasters (eg, Hurricane Sandy, Newtown) underscore the value of social ties in supporting community recovery (ie, social connectedness). In the following sections, we briefly describe the research in each core area.
Individuals on the scene are often the “first responders” to an incident and are capable of mobilizing quickly to prevent many negative impacts of disaster (eg, injury or death)7–9; thus, fostering individual and community self-sufficiency is key to minimizing the public health impact of an event. Creating a more self-sufficient population can maximize a community's ability to leverage local resources and help minimize the deployment of external aid, in many instances.10 Individual stockpiling and household emergency planning efforts are measures of self-sufficiency at the individual level, whereas postevent measures of requests for (and deployment of) external assistance indicate whether a community was self-sufficient. Current activities to increase self-sufficiency can mitigate certain negative impacts of disaster; therefore, measuring these activities and actions could help emergency planners gauge the level of resources that may be needed postdisaster and prioritize which behaviors and activities to further incentivize.
Effective partnerships between and among governmental and nongovernmental organizations (eg, regional public health and health care coalitions) contribute to the overall preparedness of a community by maximizing participation in preparedness activities and increasing the volume and diversity of resources (by pooling them), including personnel, land resources, and in-kind assets.11 Effective partnerships can also help shore up critical infrastructure through memorandums of agreement prior to a disaster.10 The National Health Security Strategy underscores the need for partnerships in stating, “The actions required to fully achieve community resilience are beyond the scope of a single department, policy, or level of government.”3 In addition, the CDC's Public Health Cooperative Agreement Performance Measures emphasize the importance of partnerships with 2 performance measures that capture the extent to which a diversity of organizations are involved in community preparedness and recovery efforts.12 Measures that assess both the diversity of participation, such as the CDC measures, and the quality of the partnership can suggest where additional partners need to be engaged as well as where existing partnerships need to be strengthened.
Social connectedness, or the interconnectivity of individuals and organizations in a community, contributes to a community's overall preparedness.10 These personal and professional social networks can be accessed by individuals in need of information and real or perceived social supports, as well as leveraged to quickly mobilize needed resources after a disaster.13–15 Living in a community with strong social connectedness can improve survival chances and safety of community residents during a disaster.16,17 Measuring social connectedness could include assessments of neighbor-to-neighbor connections, connections of individuals to organizations, or connections among key service-providing organizations. Measuring both the absence/presence of such networks as well as their overall structure (eg, where are the key communicators within these networks) can provide useful direction on where to broaden the reach of emergency communications, how to leverage networks for information, and who to engage in deeper partnership.
Despite the recognition that these 3 core areas are critical to evaluating the state of community preparedness, to date, very little progress has been made on measurement. As with other areas of emergency preparedness research, the infrequency of disasters makes ongoing measure development challenging (eg, creating robust predictive models and distinguish variables that drive readiness).2
Furthermore, inconsistent definitions of “community” have also contributed to a lack of clarity about how to bound and assess community readiness or preparedness (eg, should measures be at the individual, household, organizational, or broader community level; should individual or household measures be aggregated to reflect resilience at the broader community level; and how do you define community [eg, by zip code, county]?)18 Finally, there is growing recognition that concepts such as engagement and social connectedness are ambiguous and can be difficult to define and operationalize among practitioners, making measurement challenging for researchers.
To help address these challenges in measurement and begin to set an agenda for future measure development, this study aimed to identify existing measures of partnership, self-sufficiency, and social connectedness, as well as gaps and opportunities in the measurement of community preparedness and resilience. To accomplish these aims, the study team conducted a broad environmental scan to identify a list of possible measures in these areas and then assessed these measures using a set of standard criteria and feedback from key informants. A list of measures identified through the environmental scan and an assessment of their strengths and weaknesses for use in planning and public reporting are described in the following text.
To identify existing measures of partnership, self-sufficiency, and social connectedness, first, we conducted a broad environmental scan to identify a list of all possible measures in these areas and then narrowed the list of identified measures using a standard set of evaluative criteria and feedback from stakeholders.
We reviewed national survey instruments such as the Behavioral Risk Factor Surveillance System (BRFSS) survey,19 the National Health Interview Survey,20 and the Federal Emergency Management Agency's (FEMA's) Citizen Corps survey,21 grant program guidance (eg, Hospital Preparedness Program and Public Health Emergency Program [PHEP] Cooperative Agreements),12 Healthy People 2020 measures,22 peer-reviewed and gray (organizational and practitioner) literature, and Web sites of relevant agencies and organizations such as the CDC, Department of Housing and Urban Development, and Red Cross. Published and gray literature databases that were searched included PubMed, Scopus, Google Scholar, EMBASE, CINAHL, and the New York Academy of Medicine's Grey Literature Report. As a final step, we reviewed all survey instruments and reports listed in the FEMA's Disaster Preparedness Surveys Database,23 which compiles household, business, and school surveys that explore individual or organizational preparedness. We used the following keywords to guide our searches of the literature: “(metric OR measure* OR instrument OR indicator) AND (resilience OR partnership OR engagement OR self-suffic * OR preparedness OR connectedness). While we prioritized the preparedness and health literature, we also looked more broadly to identify sources in other disciplines that may be applicable to preparedness.
Although we mapped out 8 key domains of community resilience in our previous work,10 we focused our environmental scan on measures in the areas of self-sufficiency, partnership (including the engagement of individuals and organizations), and social connectedness. We reasoned that these were focus areas of both the National Health Security Strategy (objective 1)3 and CDC's Community Preparedness Public Health Emergency Preparedness Capability.5 In addition, these domains can be measured at various levels, including the individual level, organizational level, and/or wider community level, thus illustrating the complexities regarding the appropriate unit of analysis in community resilience. The domains of community resilience that were not captured in this scan included population wellness/well-being, access to care, quality, and efficiency. We excluded these because they have not been prioritized in recent policy documents. This may be the case because these domains reflect even broader public health goals outside the scope of traditional emergency preparedness.
For the purposes of this exercise, we used the research described earlier to define self-sufficiency as the ability of individuals and communities to assume responsibility for their preparedness and limit requests for outside assistance. Partnership refers to efforts both to promote partnerships within and between government and nongovernmental organizations and to promote civic engagement among individuals and organizations. Finally, social connectedness refers to the connections between individuals and organizations within a community that ensure access to critical resources (eg, information, financial assistance) and promote mental health. Our search revealed more than 3-dozen measures, many of which have not traditionally been applied in a preparedness context.
Narrowing the list of measures
In the next phase of our scan, we sought to eliminate from the list of measures the measures that were of relatively poor quality and retain those that were of the highest to moderate quality. To accomplish this, the study team scored measures using 5 evaluative criteria: (1) validity (ie, does the measure what is intended?); (2) reliability (ie, are the data subjective or objective?); (3) utility (ie, is the measure useful to stakeholders?); (4) maturity (ie, has the measure been piloted or fielded? Is it about to be retired?); and (5) feasibility (ie, does the measure pose a reasonable burden for collection and have the support needed to sustain it?). We chose the top 25% of measures on the basis of total score, which yielded a set of 8 measures across the 3 domains of interest.
Next, we vetted these 8 most highly rated measures with a convenience sample of 9 informants representing local and state health departments and the US Department of Health and Human Services. These experts were recruited from a group of the study team's professional contacts who regularly engage on topics related to community resilience and preparedness. Informants were invited to participate in 30-minute semistructured interviews in which they were asked to review and comment on the strengths and weaknesses of the measures. We probed the 5 evaluative criteria. We also queried these informants about other measures that may be missing, including data sources that have not historically been considered for community preparedness measurement but may, in fact, be relevant. We reviewed interview notes for comments and recommendations but did not apply traditional qualitative data analysis techniques due to fact that exercise was designed to solicit informal input rather than develop or modify a theory or conceptual model.
We list and briefly describe the most robust measures of self-sufficiency, partnership, and social connectedness. We also highlight publicly available information on these measures (eg, baseline status if available) as well as summarize informant comments regarding their strengths and weaknesses.
In the area of self-sufficiency, we identified 3 measures, all of which measure individual preparedness behaviors. Two of the measures are from the FEMA's Citizen Corps survey21 (% with a plan and % with first aid training), and one is from the preparedness module of CDC's BRFSS survey (% stockpiling specific supplies).19 Informants mentioned that a major strength of these measures is that they are pulled from national surveys that have been fielded more than once in the past decade; thus, baseline data—and in some cases, data on trends—are available (Table 1). As such, they are among the more mature, “reportable” indicators of community resilience. One major limitation of the BRFSS measure, however, is that the CDC no longer plans to sponsor the preparedness module, so this question will no longer be fielded by states in 2013. Thus, this measure cannot be used to track progress toward individual preparedness in coming years. In addition, fewer than 18 states have fielded it 1 or more times since 2006. Thus, the extent to which these data can be used to tell a story about “national” preparedness or preparedness at the community level is in question. With BRFSS, it is not always possible to drill down to the county level among the few states that fielded the optional preparedness module at least once. Also, community-level analyses are not even an option for FEMA/Citizen Corps data because fewer than 5000 people across the United States are surveyed each time the instrument is fielded (3 separate times since 2003). Another major limitation of these measures is that all fall at the individual or household level and do not consider the self-sufficiency of organizations or broader systems. Because they do not inventory these organizational and system assets, these measures provide limited information about whether the broader community is self-sufficient (ie, limited reliance on outside resources during and after a disaster).
Key informants further explained that there is some controversy regarding survey questions on individuals' stockpiling and planning behaviors. For example, simply having a detailed plan does not necessarily indicate that household members have knowledge of the plan's content or could follow its instructions in an event.
In the area of partnership and engagement, we identified a total of 3 measures. All of the measures are from the CDC/PHEP and capture the activities of health departments, rather than individuals or other governmental or nongovernmental organizations. A major strength of these measures is that they are well specified in grant guidance, and they were vetted by numerous experts. Also, data on performance will be provided by all PHEP grantees across the United States, thereby providing a national snapshot. Nonetheless, because these measures are new for fiscal year 2012, we do not currently have any data and cannot report baseline levels of performance (Table 2). As described by informants, another drawback of these measures is that they focus exclusively on the activities of one major stakeholder within the community, the local health department. Because community resilience emphasizes a whole-community approach,2 these measures are somewhat limited.
We identified 2 measures of individuals' ties to other individuals in their communities including percentage of individuals who talk to their neighbors and percentage who exchanged favors with their neighbors. While these measures do not have an obvious link to emergency preparedness in the traditional sense, key informants argued that such measures of social connectedness are crucial for a range of health outcomes including ability to withstand and recover from an incident. Because neighbors are often called upon to support one another during an event (when external assistance is not yet available) and are the true “first responders,” communities with strong neighbor-to-neighbor connections will be more self-sufficient and thus more resilient.
A major strength of both of these measures is that they are from the US Census Bureau's Current Population Survey (Table 3), a national survey of approximately 150 000 individuals. This is a large enough sample to conduct sophisticated statistical analyses, in some cases at the local level.24
Our results suggest that while there is great interest in quantifying community preparedness, existing data have major limitations. Robust indicators of community preparedness are few in number, and even when data are collected, the information may not always be tracked systematically to allow for both a national assessment and a localized picture to drive community planning.
At present, the strongest measures for the purposes of public reporting and planning are the CDC/PHEP community partnership measures. Data on these specific measures should be available in the next 12 to 18 months, although future analysis should explore the association of these variables with real and exercised responses. For many of the older measures, baseline data are available; however, these measures are focused exclusively on the activities of individuals. It is unclear whether rolling up these data to the community level (eg, % of individuals within a county) is a meaningful predictor of community response and recovery capacity. Furthermore, the social connectedness items from the Current Population Survey have not been used in the emergency preparedness context.
By cataloging measures of self-sufficiency, partnership, and social connectedness, we revealed important gaps, namely, the need for more measures of organizational and network activity and measures in the area of community engagement and empowerment. There are also no true capability measures within this set; ideally, we should have a mix of both capacity and capability measures.25 While the CDC/PHEP community partnership measures are strong because they capture interactions that are occurring among numerous sectors in the community, they are only measures of “who is at the table” rather than measures of the strength, effectiveness, or quality of those relationships. Once these measures are tested across the United States by PHEP awardees, the CDC should consider adding additional measures that focus on how such relationships are being leveraged to achieve preparedness outcomes.
Our results also suggest opportunities for how and when community preparedness data are collected. For example, we do not have clear expectations for what communities can capture during a real-time event and what preevent information could be in place to aid that analysis. In addition, there are currently no systematic frameworks regarding how information on community-level sufficiency, partnerships, and social connectedness will guide intervention or trigger local decisions about where to build community capacity.
The major limitations of our environmental scan include (1) lack of a systematic approach to identifying nonhealth sources; (2) subjective approach to scoring identified measures; and (3) use of a convenience sample of informants who represent only the health sector rather than the full range of sectors engaged in community preparedness efforts; however, to our knowledge, this is the first study to undertake such an ambitious scan of the peer-reviewed and gray literature to catalogue existing measurement efforts.
Although public health practitioners and policy makers can use the measures described here to begin to quantify and assess trends on select aspects of community preparedness and disaster resilience, a great deal of additional work is necessary. Ongoing measurement facilitates public reporting and accountability, as well as continuous quality improvement; however, these purposes cannot be achieved unless we develop new measures in areas where there are gaps and test novel measures that have not historically been applied to the preparedness context. We anticipate that quantifying resilience will be as complex as achieving it.
1. Uscher-Pines L, Chandra A, Acosta J, Kellermann A. Citizen preparedness for disasters: are current assumptions valid? Disaster Med Public Health Prep. 2012;6(2):170–173.
2. Committee on Increasing National Resilience to Hazards Disasters. Disaster Resilience: A National Imperative. Washington, DC: National Academies Press; 2012.
4. Federal Emergency Management Agency. National Disaster Recovery Framework: Strengthening Disaster Recovery for the Nation. Washington, DC: US Department of Homeland Security; 2011.
6. Chandra A, Acosta J, Meredith L. Understanding Community Resilience in the Context of National Health Security a Literature Review. Santa Monica, CA: RAND Corporation; 2010.
7. Jacob B, Mawson A, Payton M, Guignard J. Disaster mythology and fact: Hurricane Katrina and social attachment. Public Health Rep. 2008;123(5):555–566.
8. AufderHeide E. Common misconceptions about disasters: panic, the “disaster syndrome,” and looting. In:O'Leary M, ed. The First 72 Hours: A Community Approach to Disaster Preparedness. Lincoln, NE: iUniverse Publishing; 2004:340-380.
9. Helsloot I, Ruitenberg A. Citizen response to disasters: a survey of literature and some practical implications. J Contingencies Crisis Manag. 2004;12(3):98–111.
10. Chandra A, Acosta J, Stern S. Building Community Resilience to Disasters: A Way Forward to Enhance National Health Security. Santa Monica, CA: RAND Corporation; 2011.
11. Norris F, Stevens S, Pfefferbaum B, Wyche K, Pfefferbaum R. Community resilience as a metaphor, theory, set of capacities, and strategy for disaster readiness. Am J Community Psychol. 2008;41(1):127–150.
12. Centers for Disease Control and Prevention. Public Health Emergency Preparedness (PHEP) Cooperative Agreement: Performance Measure Specifications and Implementation Guidance. Atlanta, GA: Centers for Disease Control and Prevention; 2011.
13. Putnam R. Bowling Alone: The Collapse and Revival of American Community. New York, NY: Simon & Schuster; 2001.
14. Magsino S. Applications of Social Network Analysis for Building Community Disaster Resilience: Workshop Summary. Washington, DC: National Academies Press; 2009.
15. Yong-Chan K, Jinae K. Communication, neighbourhood belonging and household hurricane preparedness. Disasters. 2009;34(2):470–488.
16. Buckland J, Rahman M. Community-based disaster management during the 1997 Red River flood in Canada. Disasters. 1999;23(2):174–191.
17. Schellong A. Increasing Social Capital for Disaster Response Through Social Networking Services (SNS) in Japanese Local Governments. Arlington, VA: National Science Foundation; 2007.
18. Uscher-Pines L. New measures of community preparedness and community recovery: opportunities and challenges. Paper presented at: Dynamics of Preparedness; 2012; Pittsburgh, PA.
21. Federal Emergency Management Agency. Personal Preparedness in America: Findings From the Citizen Corps National Survey. Washington, DC: Community Preparedness Division, Federal Emergency Management Agency; 2009.
25. Nelson C, Beckjord E, Dausey D, Chan E, Lotstein D, Lurie N. How can we strengthen the evidence base in public health preparedness? Disaster Med Public Health Preparedness. 2008;2(4):247–250.