Journal of Public Health Management & Practice:
Decision-Support: Research for Policy Making
Burke, Donald S. MD; Grefenstette, John J. PhD
Public Health Dynamics Laboratory (Dr Grefenstette), Graduate School of Public Health (Dr Burke), University of Pittsburgh, Pittsburgh, Pennsylvania.
Correspondence: Donald S. Burke, MD, Graduate School of Public Health, University of Pittsburgh, 130 DeSoto St, Pittsburgh PA 15261 ( firstname.lastname@example.org; email@example.com).
The Public Health Dynamics Laboratory's MIDAS (Models of Infectious Disease Agent Study) National Center of Excellence receives funding from the National Institutes of Health's National Institute of General Medical Sciences MIDAS grant U54GM088491. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
The authors declare no conflicts of interest.
Public health emergencies such as epidemics (and other natural or anthropomorphic disasters) are complex, dynamic processes, with many interacting subsystems. Preparing for and responding to such a complex event requires deep insight into the fundamental system dynamics. However, as Leischow and Milstein1 pointed out regarding public health practice in general, “Conventional forms of problem framing, action planning, and evaluation often exclude or ignore precisely those features of dynamic complexity that make public health challenges so formidable and public health responses so innovative.”(p403) The articles in this issue resulted from a Conference on the Dynamics of Preparedness that focused on the opportunities and challenges in bringing computational models to bear on problems in preparedness and emergency response. The University of Pittsburgh organized this conference through the Pittsburgh MIDAS (Models of Infectious Disease Agent Study) National Center of Excellence, which is funded by the National Institutes of Health/National Institute of General Medical Sciences ( www.midas.pitt.edu). In this note, we emphasize a few themes that we think lay a foundation for sound research on modeling for preparedness. In particular, we believe the following:
- Explicit computational representations of complex systems can provide valuable support to decision making.
- The modeling process is often a useful exercise in itself.
- Building a model is just one step, albeit and important one, in a “meta-modeling” process that involves data acquisition, data curation, statistical inference and pattern detection, model development, representation of human behavior, consideration of pragmatics such as logistics, cost-benefit analyses, visualization, and communication of model results.
- The first link in the meta-modeling chain is reliable real-world data, which ideally should be easily accessible, computable, and disaggregated (ie, individual level).
- Efforts should be made at every jurisdictional level (county, state, country, world) to strengthen the entire “Public Health Meta-modeling Pathway” in order to support sound decision making for preparedness.
The following subsections present these points in further detail.
The Value of Explicit Models
As Jay Forrester2 explained:
All decisions are made on the basis of models. Most models are in our heads. Mental models are not true and accurate images of our surroundings, but are only sets of assumptions and observations gained from experiences ... Computer simulation models can compensate for weaknesses in mental models.
Modeling is a key step in any decision-making process; better models lead to improved knowledge and wiser decision making for public health. Models also allow for evaluation of interventions that cannot easily be tested in the real world because of a variety of constraints3 and to consideration of counterfactual circumstances that might never occur in real life.4–6 Computational models that accurately reflect the underlying mechanisms of complex systems can provide insights into dynamic relationships among real-world variables that include nonlinear interactions and feedback loops that may lead unexpected long-term outcomes.
The Value of the Modeling Process: Systems Thinking
When undertaken as a serious collaboration with decision makers, the very process of creating the model is often useful in helping decision makers realize shaky assumptions that may be hidden within their own mental models (Figure 1). The imposed discipline that comes with the requirement to define the key mechanisms that drive the dynamics of a complex system (such as a response to an emergency) fosters systems thinking, which might be defined as the “ability to recognize system-level properties that result from dynamic interactions among human and social systems and how they affect the relationships among individuals, groups, organizations, communities, and environments.”10(p23)
Not Just Model, the Whole Meta-model
When we began to work on epidemic modeling, we put most of our energy into building the models themselves. Now we spend at least as much time on activities that are not modeling per se but that are nonetheless necessary for effective modeling and decision support. Collectively, we call these activities the “meta-model” (Figure 2). Alternative terms for this concept include the “integrated information supply chain,” or the “information ecosystem.” Public health meta-modeling activities include data acquisition, data curation, statistical inference and pattern detection, model development, representation of human behavior, consideration of pragmatics such as logistics7 and legal structures,8 cost-benefit analyses, visualization, and communication of model results.
The Importance of “Ground Truthing” With Real-World Data
The first steps in the meta-modeling process are data acquisition, collection, and curation. Data are needed to support statistical analyses that identify patterns in real systems, to calibrate the individual mechanisms in the model, and to validate and test the model. We strongly endorse calls for transparent modeling practices9 including open access to both models and data so that results can be replicated by other modelers. It follows that requirements for data include that they are accessible, computable, and disaggregated.
Access to real-world data is often a limiting factor in applying modeling to preparedness, due to a lack of well-supported databases that capture data describing past disasters. An ideal database would include detailed spatiotemporal data describing historical instances of similar events, including measures undertaken to protect public health, and quantitative outcome reports. We believe that efforts should be made at every level (county, state, country, world) to strengthen and modernize the information flow in the public health decision support system.
We strongly endorse the view that all decisions are based on models and that computational models can help compensate for weaknesses in mental models. Computational modeling works best—and decision making works best—when modelers and the decision makers work closely together on the design and use of models. While modeling is a central step in the decision support process, there are both upstream data and information requirements and downstream pragmatics and economic calculations needed to help support good decision making. The meta-model should be actively designed by interactions between modelers and decision makers. Efforts should be made at every level (county, state, country, world) to strengthen the entire public health meta-modeling pathway in order to support good decision making for preparedness.
As a step toward the broader application of computational modeling in public health, The University of Pittsburgh Public Health Dynamics Laboratory is developing a set decision support computational tools, including a database for gathering and analysis of public health data (TYCHO), a large-scale agent-based epidemic modeling framework (FRED), logistic analysis and supply chain modeling (HERMES), and a public health visualization tool (GAIA), and others. We believe that supporting the entire meta-modeling pathway will facilitate better planning and preparedness for a variety of future threats to public health.
1. Leischow SJ, Milstein B. Systems thinking and modeling for public health practice [published online ahead of print January 31, 2006]. Am J Public Health. 2006; 96:(3):403–405.
2. Forrester JW. Learning through system dynamics as preparation for the 21st century. Paper presented at: Systems Thinking and Dynamic Modeling Conference for K-12 Education; June 27–29, 1994; Concord, MA.
3. Brown ST, Tai JH, Bailey RR, et al. Would school closure for the 2009 H1N1 influenza epidemic have been worth the cost? A computational simulation of Pennsylvania. BMC Public Health. 2011; 11:353
4. Burke DS, Epstein JM, Cummings DA, et al. Individual-based computational modeling of smallpox epidemic control strategies. Acad Emerg Med. 2006; 13:(11):1142–1149.
5. Longini IM Jr, Halloran ME, Nizam A, et al. Containing a large bioterrorist smallpox attack: a computer simulation approach [published online ahead of print August 8, 2006]. Int J Infect Dis. 2007; 11:(2):98–108.
6. Halloran ME, Ferguson NM, Eubank S, et al. Modeling targeted layered containment of an influenza pandemic in the United States. Proc Natl Acad Sci U S A. 2008; 105:(12):4639–4644. doi:10.1073/pnas.0706849105.
7. Assi TM, Rookkapan K, Rajgopal J, et al. How influenza vaccination policy may affect vaccine logistics [published online ahead of print April 24, 2012]. Vaccine. 2012; 30:(30):4517–4523. doi:10.1016/j.vaccine.2012.04.041.
8. Potter MA, Brown ST, Cooley PC, et al. School closure as an influenza mitigation strategy: how variations in legal authority and plan criteria can alter the impact. BMC Public Health. 2012; 12:(1):977
9. Caro JJ, Briggs AH, Siebert U, Kuntz KM.; ISPOR-SMDM Modeling Good Research Practices Task Force. Modeling good research practices—overview: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force-1. Value Health. 2012; 15:(6):796–803. doi:10.1016/j.jval.2012.06.012.
10. ASPH Education Committee. Master's degree in public health core competency development project. Version 2.3. www.asph.org
. Published August 2006; .