Kirkwood, Amy MS; Guenther, Eric MPH; Fleischauer, Aaron T. PhD, MPH; Gunn, Julia MPH, RN; Hutwagner, Lori MS; Barry, M. Anita MD, MPH
Amy Kirkwood, MS, is Bioterrorism Program Coordinator at the Communicable Disease Control Division, Boston Public Health Commission, Boston, Massachusetts.
Eric Guenther, MPH, is an epidemiologist with the Bioterrorism Preparedness and Response Program, National Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia.
Aaron T. Fleischauer, PhD, MPH, is the team leader of Epidemiology Surveillance and Response, Bioterrorism Preparedness and Response Program, National Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia.
Julia Gunn, MPH, RN, is Associate Director of the Communicable Disease Control Division, Boston Public Health Commission, Boston, Massachusetts.
Lori Hutwagner, MS, is a statistician with the Epidemiology Surveillance and Response, Bioterrorism Preparedness and Response Program, National Center for Infectious Diseases, Centers for Disease Control and Prevention, Atlanta, Georgia.
M. Anita Barry, MD, MPH, is Director of the Communicable Disease Control Division, Boston Public Health Commission, Boston, Massachusetts.
Corresponding author: Julia Gunn, MPH, RN, Communicable Disease Control Division, Boston Public Health Commission, Boston, MA 02118 (e-mail: email@example.com).
In 1999, the Centers for Disease Control and Prevention (CDC) began allocating bioterrorism funds for public health epidemiologic and surveillance capacity development. Since 1999, federal funds in excess of $650 million have been directed to state and local health departments specifically for the enhancement and development of traditional and nontraditional surveillance systems through the Cooperative Agreement on Public Health Preparedness and Response to Bioterrorism. Guidance for the use of these funds has included the implementation and operation of syndromic surveillance systems.1,2 Conservative estimates to date suggest that syndromic surveillance systems have been implemented in more than 200 state and local health departments in the United States (L. Hutwagner, MS, Statistician, CDC, personal communication, March 2006). These systems vary widely with respect to the methods used to collect and analyze data. However, the goals are consistent across systems: to detect potential outbreaks using automated, prediagnostic data sources (eg, emergency department [ED] chief complaints, over-the-counter medication purchases).3 Signals produced using aberration detection methods may lead to a faster public health response and a consequent reduction in morbidity and mortality.4
Although cost continually exists as a mitigating factor for public health activities, the direct costs for implementing and maintaining syndromic surveillance have not been fully elucidated. To date, only one domestic local health department has published a crude estimate of annual direct maintenance costs of their syndromic surveillance system—the New York City annual expenditure is estimated at $130,000 or $150,000 per year.5,6 Internationally, an evaluation of the National Health Service direct syndromic surveillance system used in England and Wales estimated the direct cost of the system to be $280,000 per year.7 In light of decreasing federal funds, calculating direct costs associated with the development, implementation, and operation of syndromic surveillance systems is a critical public health responsibility,8 and little published data are available.
Presenting direct costs of a local syndromic surveillance system may help to guide public health decision making. The Boston Public Health Commission (BPHC), in collaboration with CDC, and Massachusetts Department of Public Health (MDPH) developed and implemented an electronic ED-based syndromic surveillance system prior to the 2004 Democratic National Convention (DNC), with the immediate goal of enhancing mass gathering surveillance and a long-term goal of improving the city's overall communicable disease surveillance capabilities. In this article, we estimate the direct cost of syndromic surveillance in Boston in two phases: development/implementation and operation. In addition, public health policy and regulations relevant to implementing syndromic surveillance systems are discussed, as are costs associated with technical and scientific upgrades required to improve system efficiency.
Boston's Syndromic Surveillance System
Through funding made available by MDPH from a CDC Preparedness Grant, BPHC began development of a syndromic surveillance system in late 2003 as part of preparations for the 2004 DNC, a National Special Security Event.9 This automated system continues to provide data to monitor chief complaints of individuals presenting at 10 hospital EDs in Boston as a means of detecting unusual patterns that may predict communicable disease cases or outbreaks. Collected data, including a patient identifier (which is typically a billing or visit number that can only be connected to the patient medical record number by the hospital), chief complaint, age, sex, race/ethnicity, and zip code of primary residence, are transmitted electronically every 24 hours to BPHC. Initially, chief complaints were classified into seven infectious disease–related syndrome categories, but have been expanded to include noninfectious syndromes. Syndromes are analyzed using the Early Aberration Reporting System.10 BPHC epidemiologists review daily syndrome counts, stratified by hospital, neighborhood, age, and gender, for parameters indicative of a cluster or a clinically significant event. If a cluster or clinically significant outcome is identified, medical records are obtained and reviewed by BPHC staff, and further action is taken using standard public health methods.
The system, which is built on an existing ED volume-based surveillance system, was developed between December 2003 and June 2004, was implemented in July 2004, and continues to be used daily by BPHC. Research and development to improve functionality and signal detection capability is ongoing and has included upgrades such as the creation of a chief complaint data dictionary. In addition to data from EDs, information from daily review of death certificates, Boston Emergency Medical Services (EMS) calls, and Boston-based calls to the Regional Center for Poison Control and Prevention are incorporated into syndromic surveillance to improve situational awareness. This analysis calculates all direct costs associated with the syndromic surveillance system between December 2003 and July 2005.
Internal documents were reviewed to obtain cost data at BPHC and CDC, National Center for Infectious Diseases, Bioterrorism Preparedness and Response Program. Costs incurred by participating hospitals were not included; however, all hospitals routinely collect these data elements in their electronic information systems used for the delivery of emergency healthcare services. CDC costs were included in these calculations because it would have been necessary for BPHC to hire additional external consultants for development of programs for data processing and aberration detection algorithms had CDC personnel not been available.
Data collection included fiscal record review, internal interviews, and time studies for the period between December 2003 and July 2005. Documentation was collected on planning meetings with hospital partners and others such as MDPH that were involved in the development and implementation process. Equipment purchase and salary data for BPHC and CDC were collected. Key administrative and program staff in the BPHC Communicable Disease Control Division, Information Technology (IT) Division, and Fiscal Department were interviewed to estimate person hours devoted to the syndromic surveillance system. Logs of daily syndromic surveillance activity were analyzed to determine the frequency with which further investigation of signals or unusual cases was performed. BPHC IT logs, routinely kept to assess IT effort and expenditure, were reviewed to assess system-related activities. Four external consultants who provided IT services were included with personnel costs. In addition, CDC representatives traveled to Boston on four occasions to provide technical support. Costs included personnel wages consistent with the time of travel, airfare, rental car, and travel per diem.
This analysis focused on direct costs associated with the syndromic surveillance system in Boston. Direct costs are most commonly defined as costs that can be related directly to the production of a product or a service. Costs were divided into development, operating, and upgrading costs (Figure 1) and defined as fixed or variable. Fixed costs were defined as “one time costs” or those that do not vary with use (eg, software, equipment). Variable costs were those that vary with changes in volume of activity (eg, person time).11 Development costs were those that associated with buying equipment (eg, computer hardware, software systems), the development of regulations required to move the project forward, and personnel costs. Operating costs were those that associated with daily system maintenance, and were composed of variable costs (person time = hourly wage × time required to complete function) and fixed equipment costs (eg, computer software licensure). Upgrading costs were those that associated with any improvements made to the system throughout the 19-month period (eg, person time). BPHC and CDC personnel costs included both salary and fringe benefits (32.3%). The development and operating phases were defined as December 1, 2003–June 30, 2004, and July 1, 2004–July 1, 2005, respectively. Development took place during the 7 months leading up to the DNC in Boston, which occurred in late July 2004. The perspective given is that of a local, state, and federal government operating budget, where costs were allocated to the year in which they occurred.
Table 1 shows the total investments made by BPHC and CDC. Over the 19-month period of analysis, the direct cost to the BPHC was $396,716. CDC made an investment of $26,183 during the same period. The total cost associated with syndromic surveillance system development, operation, and upgrade was $422,899.
Costs for BPHC during the development phase were $126,025, including $22,140 for equipment and $103,885 for personnel. CDC made an investment of $15,202 during the development phase. Equipment purchases (18% of BPHC development costs) included two servers (one for daily operations and another for backup) that support Web, FTP, and SAS application functions. Computer software included an Oracle Corporation Software License and Service agreement (Oracle, Redwood Shores, California), a SAS Institute Inc (Version 8, Cary, North Carolina) License Agreement, and the Navicat Application (PremiumSoft, Hong Kong SAR). Other equipment costs were for miscellaneous computer equipment and supplies: one laptop, one desktop computer, a PC wireless card, a monitor switch, and a T1 Line installation and service. Of BPHC personnel costs, 11 percent were associated with the development of a local health department regulation including drafting a regulation and guidelines, holding meetings for public comment, and presenting the regulation to the BPHC Executive Board for approval.
Operating costs for BPHC were $139,037, with personnel accounting for 53 percent of the expense. Staff spent time reviewing and following up on incoming data and working on IT issues to ensure data quality control and maintenance of the SAS system. Expanded follow-up of signals by the epidemiologist and public health nurses, including contacting hospital ED staff to investigate specific cases, occurred on average three times a month. Electronic health alerts, developed and sent to multiple healthcare providers and institutions in Boston in response to syndromic surveillance findings, required additional personnel time. Equipment costs were for an emergency backup phone line (T1 Line), to ensure all servers were functioning effectively, and software licenses. Costs incurred by CDC were related to travel for two CDC representatives to be on-site in Boston during the DNC to assist with syndromic surveillance.
A major system upgrade was the review of 60,000 medical records to develop a “dictionary” of key words or phrases used in chief complaints to improve accuracy and flexibility of syndrome definitions. In addition, the technology interface of the system was improved to allow more efficient data access. During this phase, BPHC costs were $57,265. CDC costs of $2,526 were for travel to Boston for training of BPHC personnel on IT system upgrades.
Syndromic system enhancements
To improve the ability of BPHC to interpret data, enhancements to the basic syndromic surveillance system included obtaining 2 years of historical data from nine hospitals, plus historical laboratory data from two of the nine hospitals. The availability of historical baseline data has facilitated the validation of syndrome definitions and the understanding of communicable diseases such as influenza that occur annually. BPHC also developed a contract with the Regional Center for Poison Control and Prevention to receive information daily on calls from Boston-based providers. In addition, data from all death certificates issued in the city of Boston and from 911 calls to Boston EMS are sent electronically to BPHC daily and used in the interpretation of information submitted from EDs. Costs associated with these system enhancements amount to $74,389.
To date, the costs associated with syndromic surveillance systems have not been described in detail. In this study, the total direct cost of developing, implementing, and using a local health department syndromic surveillance system over a 19-month period was $422,899. Prior studies that focused on operating costs found annual costs to range from $130,000–$150,0005,6 to $280,000.7 BPHC operating cost (including necessary upgrades) of $196,302 over a 12-month period falls within this range. Enhancements made to the Boston system ($74,389) may not be necessary in all jurisdictions that operate a syndromic surveillance system, but in Boston they provide supporting information used in the interpretation of ED-linked data.
This study focused on the costs of a syndromic surveillance system in a large metropolitan area. Costs in other public health jurisdictions may differ. In addition, health departments seeking to operate a system similar to that in Boston must also consider costs associated with continued use, such as recurring costs and replacement costs. Other limitations include the method in which person time was determined. Because staff members were interviewed to determine the amount of time they spent on the syndromic surveillance, there is room for error in the form of recall bias. The syndromic surveillance system in Boston was implemented in part to enhance preparedness for the DNC. The sense of urgency related to a mass gathering designated as a National Special Security Event may have helped facilitate system implementation, thus this should be a consideration when interpreting the results. Furthermore, the BPHC had previously operated a volume-based surveillance system in which local hospitals reported ED or urgent care daily volume data. Existing relationships between public health and the medical care system may have facilitated the transition to the reporting of data elements needed for syndromic surveillance. The additional costs required to implement a syndromic surveillance system in a location without existing enhanced surveillance is unknown. The costs incurred by participating hospitals were not assessed. However, because all participating hospitals were already operating electronic information systems from which surveillance data were extracted, we estimate that the costs to these hospitals to be minimal; this may not be the case in all areas.
This study focused only on the direct costs of the syndromic surveillance system. Indirect and administrative costs (such as building costs, grants, and fiscal support costs) associated with the system at BPHC (estimated to be approximately $60,000) and costs incurred by MDPH (primarily technical consultation) were not included. In addition, this study did not perform cost-benefit analysis. However, BPHC has used the syndromic surveillance system in addressing varied public health issues. Data from the system have been used to identify increased incidences of carbon monoxide poisoning, heat-related illness among nonelderly, and influenza, and has been used during measles outbreaks. In these situations, the BPHC conveyed information to healthcare providers and others, often with the assistance of partners such as MDPH, so that steps could be taken to minimize risk.
To perform situational awareness during these events, the flexibility of the system also allows creation of syndromes using the chief complaint data dictionary, the result of the aforementioned 60,000 medical record review done by BPHC staff. For example, because of the recent outbreak of mumps in the midwestern United States, the BPHC has built a “mumps” category within the syndromic system, which selects chief complaints for “facial swelling” plus “fever,” while excluding complaints that are associated with dental problems. BPHC has also used this system (in conjunction with historic data from hospitals) to track influenza patterns, something that has proven very useful for both the health department and local providers. This public health use of syndromic surveillance has been very beneficial in certain situations, but more formal evaluations of usage of syndromic surveillance systems, such as a cost-benefit analysis where outcomes are considered, are needed. As noted by others,3,4,12 cost-benefit research is critical to determine the utility for syndromic surveillance in daily public health activities.
Incremental cost for implementation and operation of a syndromic surveillance for communicable diseases in Boston was relatively small, accounting for less than 15 percent of BPHC communicable disease budget. Furthermore, IT-based surveillance systems are expanding into other areas such as emergency management. Currently, BPHC and Boston EMS are developing a patient tracking system for mass casualty events. The cost of development, hardware, and training for this initiative is estimated at approximately $360,000.
Although syndromic surveillance was originally designed for the early detection of bioterrorism-related events, in Boston, the system has demonstrated utility for tracking other public health issues (eg, injury, asthma). Likewise, syndromic surveillance may have greater utility for postevent surveillance or disaster response scenarios to maintain situational awareness. More data however are needed to identify the utility of syndromic surveillance in day-to-day local public health practice. For health departments considering syndromic surveillance, our data documents the direct costs associated with developing, implementing, and operating an electronic ED-based syndromic surveillance system in a major metropolitan health department. Although our data show costs not to be excessive, more research is needed regarding cost-effectiveness of such a system.
For more information regarding the Boston Syndromic Surveillance System, please contact the Communicable Disease Control Division of the Boston Public Health Commission at (617) 534-5611 or firstname.lastname@example.org.
2. Centers for Disease Control and Prevention. Continuation guidance for cooperative agreement on public health preparedness and response for bioterrorism—budget year five. Available at: http://www.bt.cdc.gov/planning/continuationguidance/
. Accessed May 11, 2006.
3. Sosin DM. Syndromic surveillance: the case for skillful investment. Biosecur Bioterrorism Biodefense Strat Pract Sci. 2003;1:247–253.
4. Buehler JW, Berkelman RL, Hartley DM, Peters CJ. Syndromic surveillance and bioterrorism-related epidemics. Emerg Infect Dis. 2003;9(10):1197–1204.
5. Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, Weiss D. Syndromic surveillance in public health practice, New York City. Emerg Infect Dis. 2004;10(5):858–864.
6. Heffernan R, Mostashari F, Besculides M, et al. New York City syndromic surveillance systems. MMWR Morb Mortal Wkly Rep. 2004;53(suppl):25–27.
7. Doroshenko A, Cooper D, Smith G, et al. Evaluation of syndromic surveillance based national health service direct derived data—England and Wales. MMWR Morb Mortal Wkly Rep. 2005;54(suppl):117–122.
8. Buehler JW, Hopkins RS, Overhage JM, Sosin DM, Tong V. Framework for evaluating public health surveillance systems for early detection of outbreaks: recommendations from the CDC working group. MMWR Morb Mortal Wkly Rep. 2004;53(RR-5):1–13.
10. Hutwagner L, Browne T, Seeman GM, Fleischauer AT. Comparing aberration detection methods with simulated data. Emerg Infect Dis. 2005;11:314–316.
12. Reingold A. If syndromic surveillance is the answer, what is the question? Biosecur Bioterrorism Biodefense Strat Pract Sci. 2003;1(2):77–81.
© 2007 Lippincott Williams & Wilkins, Inc.