Research Brief Report
Assessing Performance Under Pressure
Measuring the impact of public health emergencies on the organizational performance of health departments is a critical policy question in terms of anticipating the strain and added resources required for departments to manage urgent events effectively. We introduce a method of estimating this impact by using the techniques of Bayesian network analysis to identify the influences exerted on different organizational units of 4 county health departments (department of public health) operating under the strain of the H1N1 outbreak in April-July 2009.
To build a model of the influences reported in relationships among the identified factors, we constructed a Bayesian influence diagram using a decision analysis software program, GeNIe,1 to generate the diagram image. The program is based on the Bayes theorem, which asserts that the probabilities of possible courses of action are based on observed prior actions. The influence diagram provides a visual profile of the health department's performance as a system that enables analysts to see how and why certain behaviors either supported or inhibited effective response under stress. On the basis of Bayesian network models, the diagram visualizes the probabilistic influences between all pairings of identified factors and provides a systematic method of updating probabilities for possible strategies of action as conditions change and new observations provide evidence of such change. A formal Bayesian network would include the numerical probabilities that describe the level of influence between each factor, whereas an influence diagram identifies only the pattern of relationships, not their strengths.
The influence diagram was developed from data reported for the 4 county health departments. It groups the factors affecting overall performance into 4 segments: department of public health factors, county government factors, outside organization factors, and contextual factors. Interviews elicited data on the roles and actions of health department staff, specifically during the H1N1 response. By indicating how actors responded both to H1N1 demands and to each other's actions taken to meet these demands, these data report what behaviors are included in the diagram and show which first-order behaviors lead to second-order behaviors and what consequences ensue for overall departmental performance. From the data, researchers identified contextual factors and key patterns of response actions by actors at different levels; linked the response action patterns to the actors; and identified the dynamic influence of contextual factors and the actor groups on H1N1 response actions at the county health department level.
Each segment is interrelated with the other segments, with the influence links depicted visually. In addition, each of those 4 segments has some influence on 2 important behavioral factors identified within the departments of public health: defensive routines and organizational adaptation. As indicated by code frequencies and code content from the interview data, defensive routines and organizational adaptation behaviors strongly affect the efficiency and effectiveness in each of the public health departments' H1N1 responses.
The detailed Bayesian influence diagram visually represents all possible courses that a county health department response could take, given existing resources and constraints. Each department of public health followed a unique path through the model, producing differing responses with differing outcomes from the same factors in its H1N1 response. Comparing these differing paths through the model, we identified possible causes of major problems or successes, based on interview responses. The analysis included factors that lead to defensive routines, where members of the agency avoid communication because of increased stress from inappropriate expectations of outputs and timelines. Departments that reported defensive routines documented less effective performance than those that avoided these behaviors. Such routines generate critical disruptions in operations, as they break down the ability of an organization to mobilize response by undermining coordination and cooperation. The analysis also identified factors that promote organizational adaptation, such as increased, interactive communication among units within the department, and support effective response. The Figure presents a simplified Bayesian influence diagram that shows the direction of influence from the external public health response system, characteristics of county-level response, and focuses on communication-influenced factors. For example, declining budgets and limited staffing and contextual characteristics of the population served by the department (dark grey icons) on the internal response system of a public health department influence either the organizational adaptation or generation of defensive routines within the department. These patterns shaped the overall response of the department to the H1N1 threat (light grey icons), where communication breakdowns drive the response toward defensive routines and away from adaptation.
The 4 counties included in this study reported different paths through this diagram. As the external county health department system increased demands for response to H1N1 in counties characterized by declining budgets and limited staffing, but large, diverse populations, internal strain on the departments led to increased defensive routines and decreased overall performance in actual response. In counties with smaller populations, later surge in demand, and stronger support from other county-level organizations, public health departments adapted more quickly and effectively.
1. GeNIe, a Development Environment for Reasoning in Graphical Probabilistic Models. Pittsburgh, PA: Decision Systems Laboratory, University of Pittsburgh. http://genie.sis.pitt.edu/
. Accessed February 4, 2013.