One of the critical responsibilities of public health departments is to identify population groups vulnerable to potential hazards including both natural and human-made hazards that could negatively impact the lives, economy, environment, and property of residents living within the jurisdictions they serve. To carry out this function, town, city, county, and state public health departments are increasingly conducting hazard vulnerability and jurisdictional risk (HVJR) assessments as part of their public health emergency planning. While HVJR assessment methods can be drawn from a strong foundation of environmental hazards and social science research,1 , 2 risk modeling assessment and management,3 and epidemiology and biostatistics,4 it is often difficult for public health practitioners to apply such methods from these diverse fields to public health practice. However, since all risk assessments, regardless of discipline, must begin with mechanistic models that identify the critical components and pathways of cause and effect, public health practitioners should have an understanding of the HVJR mechanistic models underlying the computational HVJR tools they are currently using.
HVJR assessments typically involve the measurement of hazards (exposure), environmental, physical, social, and health vulnerabilities (susceptibility), and community resources available to counter the disaster impact (resilience). While there has been considerable effort by the FEMA (United States Federal Emergency Management Agency) Risk Analysis Branch in hazard vulnerability analysis, such as Hazus-MS,5 the specific focus on public health HVJR is more recent. As a consequence, several public health–specific HVJR assessment and computational tools have been developed to help public health professionals carry out their preparedness planning activities. However, the models describing the hypothesized causal relationships and pathways between hazards, vulnerabilities, risk, and outcomes underlying the calculations in these computational tools have not be adequately described or validated. Without understanding the risk models behind the tools, it is difficult for practitioners to translate their HVRJ metrics into public health preparedness planning and action.
Furthermore, from a measurement perspective, HVJR assessment tools generally require that practitioners synthesize their qualitative experiences into numeric subjective ratings corresponding to questions assumed to measure the constructs of hazard impact, vulnerability, and resilience. Objective data on hazard likelihood and severity, social and physical vulnerability, and human and economic losses are either reviewed before assigning the subjective ratings or, if applicable, entered as objective data directly into the tool. These ratings are then averaged either numerically or by a consensus approach to calculate the quantitative scales used in the HVJR equations. These equations usually require the assumption that the qualitative assessments and individual item ratings can be combined into a linear function containing either additive or multiplicative factors, are reliable and valid measures of the hazard exposure, vulnerability, and resilience constructs, and accurately represent the corresponding causal pathways of the true underlying risk of human, physical, and economic losses. Separate from the HVJR mechanistic model, the HVJR measurement model also needs to be clearly defined and adequately validated.
In epidemiologic investigations and health outcomes research and management, mechanistic and measurement modeling methods are important aspects when determining the predictors of disease risk. In this article, we describe an approach to incorporate these methods into HVJR assessment in an effort to help public health preparedness planners understand and gain awareness of the complex nature of the multisystem HVJR model, the relationships between measurement and structural components of the model, and how to potentially increase the reliability and predictive validity of HVJR analysis.
Background and Rationale
In 1979, LeChat wrote, “Studies on the health effects of disasters could show that epidemiological indices can be of value in planning preventive and relief measures and in evaluating their effectiveness.”6(p11) He emphasized that the evaluation of the impact of the public health and health-related consequences of disasters “has been handicapped by a lack of data, particularly concerning the health situation immediately after the impact.” Forty-four years later, a manuscript extolling the need for research as part of the public health emergency response referenced the desperate need for “data” and “data collection” indicating that LeChat's concerns are still quite timely today.7 In this decade of “big data” and enormous information technology capabilities, local and state public health officials are seeking to identify those data sources and analytic methods that will be most useful for planning, responding, and recovering from disasters. The foundation of quality improvement in any system is timely and accurate information and data coupled with analysis that identifies what needs to be changed and when. The ability to effectively use both qualitative and quantitative methods to evaluate the public health system response to disasters is imperative to improving planning, response, and recovery. Through increased awareness and preventive measures, the ultimate goal of an HVJR assessment is to help ensure a unified approach that will lessen vulnerability to hazards over time. Looking forward, the assessment should also be “dynamic,” defined as building precision, reliability, and predictive validity with each new hazard and disaster occurrence. This is possible only if the mechanistic models underlying the HVJR assessment are developed and well-defined and are capable of being tested and validated using observed data.
The primary impetus for this project arose from several professional and practice-based experiences, including developing instructional and technical assistance materials as part of the Centers for Disease Control and Prevention–sponsored Preparedness and Emergency Response Learning Center at the Harvard School of Public Health, conducting HVJR assessment and analyses for the State of Connecticut Health Department, and working as university-affiliated educators and researchers, as well as local and state public health officials in Massachusetts and Connecticut. We believe that the environmental, epidemiologic, and biostatistical risk analysis sciences have not adequately informed the practice of public health, and we take this opportunity to illustrate how methods used in public health research have the potential to improve public health practice.
Methods and Activities
This study used an evidence-based model-building approach, commonly referred to as “model specification,” which begins with a conceptual model outlining in broad terms, outcomes and predictors. The relationship between the independent predictor variables and the dependent outcome variables is defined in the “structural component” of the model, which also defines how the predictors relate to each other.8 To guide our conceptual model development, we adopted the general definition of risk assessment used in the FEMA Risk Branch 2012 report as follows:
A risk assessment identifies hazards and their associated risks, including threats to public health and safety, the environment, property damage, and economic loss. The assessments combine the probabilities with the consequences in a way that quantifies risk. Quantifying the risk is a powerful way to communicate the threat, determine the key factors that cause it to be high, and ultimately perform trade-off analyses to determine the most effective way to reduce, avoid, or otherwise control it.9
To form the basis for the HVJR conceptual model, we first reviewed the literature on the theoretical framework of vulnerability, hazards, and disasters and the computational tools for public health and emergency management HVJR assessments. We then broadly identified the most critical primary predictors within the HVJR model. These broad concepts were translated into specific data variables yielding the mechanistic model displayed with the structural equation modeling (SEM) graphical interface in Stata (Release 12.0, College Station, Texas). Geospatial mapping was used to inform the mechanistic model by visually examining the variability and patterns associated within county-level demographic, social, economic, hazards, and resource data. Map layers were created that maximized the separation of geographical areas, and these results were used to identify the paths (arrows) between the observed and latent variables representing the measurement and structural components of the mechanistic model. We used public access data and Esri Community Analyst online software10 both to perform the mapping visualization analysis and to hypothesize the causal pathways linking exposure and vulnerability to hazard impact. For illustrative purposes, we selected one basic, representative mechanistic HVJR model out of many more complex alternatives to serve as a generic base reference model. We used this model in Stata to derive the resulting equations and multivariate statistical solutions and tested the feasibility of the corresponding estimation procedures using simulated county-level data.
Conceptual model of HVJR
From our literature review, we found that there is a long and rich history regarding the conceptual frameworks for hazards, disasters, and vulnerabilities dating back to the work of White and Haas11 nearly 40 years ago. Cutter's12 1996 article clearly defined social vulnerabilities. The reader is referred to her extensive list of publications in this field.13 We found that all HVJR computational tools used either a questionnaire or spreadsheet format. We also found that most tools had 3 primary domains, namely, hazards, vulnerabilities, and resilience. For the mechanistic models described here, we drew heavily from the Texas Jurisdictional Risk Assessment Resources tool14 because we found it to be the most analytically complete and comprehensive for causal modeling purposes.
We adopted the following commonly used definitions for our conceptual model: The term hazard covers a wide range of events, but from an all-hazards framework, it can be most broadly defined as a condition, event, or circumstance that could lead to or contribute to an unplanned or undesirable event. Disasters are singular larger-scale events that overwhelm the local capacity to effectively respond to and recover from an event. Hazard identification determines the areas of a community that are affected by disasters, the likelihood of a disaster occurring, and its intensity and magnitude. The impact of a particular hazard requires measurement of several subcomponents, namely, type of event (eg, tornado, flood, mass shooting, bombing, radiation), likelihood (probability of occurrence or frequency), severity (magnitude and intensity), and potentially a hazard-specific impact factor. Hazard risk is the likelihood of incurring harm, or the probability that some type of injury or loss would result from the hazard event, and often varies systematically by time and geographic region. Hazard type can be organized into a hierarchical structure according to narrower or broader classifications. For example, weather-related events such as hurricanes, tornados, coastal storm surge flooding, and blizzards may be grouped under the more general class of “atmospheric events,” earthquakes and volcanic eruptions under “geological events,” and mass shootings, bombings, and terrorist attacks under “human-made events.” When performing HVJR assessment for smaller jurisdictions, these composite classifications are often necessary to increase the precision of the likelihood estimates. Vulnerability is measured through a multitude of characteristics of the population or physical environment that would increase susceptibility to a negative public health outcome, such as social and demographic characteristics or flood zones and seismic activity. Resilience is often embedded within vulnerability, but it can also be measured as a separate construct to determine the impact of available public health, emergency management, and governmental and societal resources and capabilities that could potentially mitigate negative population health consequences.
We required that to the extent possible each of the conceptual model components should not overlap or strongly correlate across the other components that determine the ultimate value of the disaster impact construct. The hypothesized relationship between predictors and outcomes can be illustrated by a common public health preventive activity such as influenza prevention. It is common practice to decrease vulnerability by reducing exposure by personal protective equipment or social distancing or by reducing susceptibility through vaccination. Furthermore, when decreasing vulnerability is not possible or is very difficult, public health programs attempt to build up resilience by fortifying access to medical care.
Mechanistic model of HVJR: SEM model specification
While different computational tools relied on different risk equations and used various methods to gather data on the variables included in those equations, all models measured disaster impact as a function of the probability and severity of a hazard, the degree of vulnerability of the populations and physical and geographical environment, and resilience as characterized by public health, health care, and emergency response resources. Hazard likelihood and severity were assessed using an ordinal qualitative descriptive measure (eg, probable, remote, extremely remote, extremely improbable), a rank-ordered measure (eg, relative to other hazard events), or a ratio-scaled measure such as a rate (number of occurrences during some specified interval of time). Estimation of likelihood was typically based upon historical data and was either subjective or objective. Since resilience could be operationalized as the positive end of vulnerability in a bidirectional measurement scale as well as by a separately defined construct, we were careful not to confound the vulnerability and resilience constructs. For example, low-socioeconomic status could be considered a measure of high vulnerability whereas higher-socioeconomic status would constitute low vulnerability and, as a consequence, higher resilience. As such, the latter should not be included as part of the resilience construct. To avoid this problem, we chose to restrict resilience to resources and capabilities specific to the public health and emergency preparedness of communities.
When testing and validating a model, one has to decide upon the intended sampling unit. We chose to consider “counties” to be the sampling unit since they typically correspond to a jurisdictional authority and are also large enough to provide a sufficient number of occurrences or cases. There are also numerous mapping resources specific to counties that can be used to assist model development. For example, while conducting HVJR assessment for the state of Connecticut, we wanted to visualize how nursing homes were distributed throughout the state since nursing homes' residents represent a particularly vulnerable population. We obtained the addresses of nursing homes in Connecticut and uploaded these to the Esri Community Analyst software so that we could superimpose the hurricane tracks and high coastal storm surge areas against these locations. As shown in Figures 1A and B, through geospatial visualization, relationships between the densities of nursing homes throughout different counties, the hurricane tracks, and coastal surge areas are more clearly revealed. The patterns observed suggested that the coastal counties are more vulnerable because of the demographics of the nursing homes and that these relationships should be accounted for in the mechanistic model.
At the most extreme level, the mechanistic model should indicate that when the probability of the Hazard (H) and Vulnerability (V) is reduced to zero, we effectively prevent the adverse health, economic, or physical losses above and beyond the baseline stochastic variation by reducing the impact of the disaster. For example, disease eradication projects, such as smallpox and polio, attempt to reduce the probability of “H” to zero. We concluded that the complexity in measuring and estimating the probabilities of specific hazards in addition to the impact on population vulnerability to health is substantial and required causal inference theory15 and structural equations with latent variable modeling16 for further development. We translated the conceptual model into a schematic of a hypothetical mechanistic model that represents a simplified subset of the measurement and structural components of HVJR for illustrative purposes (Figure 2). In this diagram, the ovals represent latent constructs that cannot be measured directly whereas the rectangles represent variables that can be measured directly. The arrows from latent constructs to observable variables indicate how the latent constructs are quantified.
As shown, the measurement of hazard, vulnerability, and resilience yields all latent variables that are measured indirectly through another set of latent variables that are measured through observable variables diagramed with rectangles and denoted by vi, The schematic shows only v 1, v 2, and vk, which are used here to represent v 1, v 2, v 3, ... vk, where k is the number of potential observable variables required to precisely, reliably, and validly quantify the latent variable. Each latent variable oval represents a scale derived from observable quantitative data (rectangles) at the lowest level of the latent variable hierarchy. In the schematic, the highest-level latent construct is Disaster Impact (level 1), which consists of 3 latent variables at level 2 (Hazard, Resilience, and Vulnerability) and several latent variables at level 3 (lowest level). For example, it is posited that a portion of the impact of a disaster can be predicted by the level 2 latent variable construct, Vulnerability, which represents the susceptibility of a given population, system, or place to harm from exposure to the hazard and directly affects the ability to prepare for, respond to, and recover from hazards and disasters. The level 3 latent Vulnerability subscale, Social, is a measure of those demographic and socioeconomic factors that increase or attenuate the impacts of hazard events on local populations.17 , 18 This Social Vulnerability latent variable is often measured through the Social Vulnerability Index (SoVI).19 SoVI synthesizes 30 demographic and socioeconomic data variables primarily from the United States Census Bureau.
The direction of the arrows in the mechanistic model diagram is important. Straight single-headed arrows represent 1-way causal influences from the variable at the base to the variable to which the arrows point. The measurement of latent variables assumes that the underlying true latent variable construct causes the values of the observed variables to change. As shown, the basic structural component in this model implies that the higher the Disaster Impact, the higher the human, economic, and physical losses. Because of space limitations for the mechanistic model depicted in Figure 2, we selected only a few hazards, although the full model would display all. Also, we have not included the error terms for the measurable and latent variable variables, nor have we drawn bidirectional arrows that would indicate the covariance present among the different observable and latent variables. Another simplifying assumption is that we have represented level 1 and 2 latent variables to be 1-dimensional, while it is quite likely that the lower-level latent variables might not load primarily on one overall factor.
Statistical estimation methods
Estimation is the logical next step in the modeling process after model specification. We generated simulated data so that we could evaluate the feasibility of using the SEM statistical estimation procedures to validate HVJR models. Briefly, for each latent variable Xjm, where m represents the level 2 latent variable index (eg, here m = 1, 2, and 3 for Hazard, Resilience, and Vulnerability, respectively) and j the level 3 latent variable index within level 2, a corresponding set of equations for the measurement component exists such that
for each observable variable vijm and corresponding error term [Latin Small Letter Open E]ijm, i = 1, 2, 3, ..., k. The βs represent the path coefficients. A similar form of this equation exists to describe the relationship between level 3 and level 2 latent variables and level 2 latent variables and level 1 (Disaster Impact) latent variable, denoted by X. Finally, the structural component between Disaster Impact X and the outcomes Y (human, economic, and physical losses, i = 1, 2, and 3, respectively) is given by Yi = αi + Xβi + εi.
The simulated model included data items for each hazard (eg, blizzard), with ratings for likelihood, severity, and impact scored from 0 (lowest) to 10 (highest). The 2 resilience scales (Public Health and Other Sectors) each had 3 data items for ratings on capabilities, resources, and staffing, whereas each of the vulnerability scales (Economic, Social, Demographic, Physical Environment, and Health) also allowed for 3 data items specific to the construct and also ranging between 0 and 10. The data were simulated to represent hypothetical 192 counties with varying ratings for the individual items and also to yield internal consistency α coefficients of greater than .70 for level 3 latent variables. The estimation results demonstrated that it is feasible to construct the HVJR database from commonly used measures available in HVJR computational tools and to estimate the parameters of the model in Figure 2 using either maximum likelihood or asymptotic distribution free methods; however, actual county data will be necessary to evaluate model fit and select the most appropriate model.
In this project, our purpose was not to develop another computational tool for HVJR assessment but rather to demonstrate that mechanistic, geospatial, and multivariate statistical models and methods could be used to aid the public health professional better understand the complex nature, structures, and interrelationships underlying HVJR assessment. Moreover, we wanted to build educational awareness for the future adoption of dynamic, predictive estimation methods, and risk analyses. From a scientific perspective, we demonstrated that the underlying models and measurements involved in HVJR assessments and analysis are extremely complex. Measuring these sophisticated constructs relies heavily on accurate information, data, and statistics, much of which can be obtained from a number of different publically available demographic, medical, health, social, environmental, atmospheric, and geospatial databases. Current HVJR computational tools translate this complex measurement structure into subjective ordinal rating scales while summating or multiplying these ratings to obtain a metric, often without identifying or verifying the causal models underlying this inference. To achieve a deeper comprehension of how the data are used to create the latent variables and to build a model for subsequent parameter estimation, we found geospatial mapping to be helpful. Geospatial mapping redirects one's attention to the fact that the measures of hazard, vulnerability, and resilience (disaster risk) and the losses are reported for a particular geographical region such as a county.
While the quantitative methods associated with complex analytic and measurement methods, causal inference, and decision theory are common in biostatistical, epidemiologic, and health services research, they have not been adequately explored or used in public health preparedness and mitigation planning. In this study, we illustrated how measurement and causal modeling methods used in public health and medical research could be applied to public health preparedness planning and evaluation. We completed the first step of SEM, commonly referred to as “model specification,” as well as the initial steps of estimation using simulated data. The next step will be to construct and validate the HVJR model using data regions of the United States consisting of 200 counties or more. Once a refined model is found that adequately represents the data, public health practitioners from individual counties could use the model as a decision tool to predict how increasing resources or lowering vulnerability could potentially positively or negatively affect human, economic, and physical disaster–related outcomes.
We recommend that HVJR assessment training include an understanding of the complex set of relationships between the wide range of potential predictor and outcome variables as well as the intricate measurement models required for reliable and valid measures of disaster impact. Well-developed mechanistic models could be used in conjunction with existing HVJR tools initially for educational training and, ultimately, as a method for carrying out dynamic, predictive analysis. The methods proposed here could also be used to facilitate health systems quality improvements to enhance the recovery process and contribute to the resiliency of disaster-affected regions and their communities to withstand future public health threats.