Share this article on:

Economic Review of Immunization Information Systems to Increase Vaccination Rates: A Community Guide Systematic Review

Patel, Mona MPH; Pabst, Laura MPH; Chattopadhyay, Sajal PhD; Hopkins, David MD, MPH; Groom, Holly MPH; Myerburg, Stuart JD; Morgan, Jennifer Murphy MSPHthe Community Preventive Services Task Force

Journal of Public Health Management and Practice: May/June 2015 - Volume 21 - Issue 3 - p 253–262
doi: 10.1097/PHH.0000000000000100
Research Brief Report

Context: A recent systematic review found that use of an immunization information system (IIS) is an effective intervention to increase vaccination rates. The purpose of this review was to evaluate costs and benefits associated with implementing, operating, and participating with an IIS. The speed of technology change has had an effect on costs and benefits of IIS and is considered in this review.

Evidence Acquisition: An economic evaluation for IIS was conducted using methods developed for Community Guide systematic reviews. The literature search covered the period from January 1994 to March 2012 and identified 12 published articles and 2 government reports.

Evidence Synthesis: Most studies involving cost data evaluated (1) system costs of building an IIS and (2) cost of exchanging immunization data; most economic benefits focused on administrative efficiency.

Conclusions: A major challenge to evaluating a technology-based intervention is the evolution that comes with technology improvements and advancements. Although the cost and benefit data may be less applicable today due to changes in system technology, data exchange methods, availability of vendor support, system functionalities, and scope of IIS, it is likely that more up-to-date estimates and comprehensive estimates of benefits would support the findings of cost savings in this review. More research is needed to update and address limitations in the available evidence and to enable assessment of economic costs and benefits of present-day IIS.

The purpose of this study was to evaluate costs and benefits associated with implementing, operating, and participating with an immunization information system.

Community Guide Branch, Division of Epidemiology, Analysis, and Library Services (Mss Patel and Murphy Morgan and Drs Chattopadhyay and Hopkins), and Immunization Services Division, National Center for Immunization and Respiratory Diseases (Ms Pabst and Mr Myerburg), Centers for Disease Control and Prevention, Atlanta, Georgia; and Kaiser Permanente Center for Health Research (Ms Groom), Portland, Oregon.

Correspondence: Mona Patel, MPH, Community Guide Branch, Division of Epidemiology, Analysis, and Library Services, Centers for Disease Control and Prevention, 1600 Clifton Rd, MS E-69, Atlanta, GA 30329 (iwi2@cdc.gov).

Names and affıliations of Task Force members are available at www.thecommunityguide.org/about/task-force-members.html.

This review would not have been possible without the subject matter expertise and contributions of Daniel Martin and Gary Urquhart from the Immunization Information Systems Surveillance Branch at Centers for Disease Control and Prevention. The authors also thank Kate W. Harris, Kristen Folsom, and Onnalee Gomez for their assistance throughout the review process.

The authors declare no conflicts of interest.

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License, where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially.

Back to Top | Article Outline

Context

Vaccines are considered one of the most effective prevention tools used within public health to prevent vaccine-preventable diseases (VPD) among children, adolescents, and adults. Through vaccinations, the United States has experienced declines in incidence of morbidity, disability, and mortality from VPD.1,2 Recommended childhood vaccinations that protect against diphtheria, tetanus, pertussis, Haemophilus influenza type b (Hib), poliovirus, measles, mumps, rubella, hepatitis B, varicella, hepatitis A, pneumococcal, and rotavirus prevent approximately 20 million disease episodes and 42 000 premature deaths, resulting in estimated net savings of $68.8 billion (2009 dollar) from averted medical costs and reduced absenteeism from work.3

Although the success of childhood vaccination programs has led to more than 95% decline in these infectious diseases, cases of illness and death by VPD still occur, as evidenced by the measles outbreak during 1989-1991 attributable to unvaccinated or undervaccinated persons.4–6 Immunization programs and vaccination providers remain challenged to identify unvaccinated and undervaccinated subpopulations and individuals, ensure that individuals are appropriately vaccinated, and conduct outreach and interventions to maintain high vaccination coverage levels.

Use of an immunization information system (IIS) is one strategy that can be adopted to increase and improve vaccination delivery in the United States. Immunization information systems are confidential, population-based, computerized databases that record all vaccination doses administered by participating providers to people residing within a given geopolitical area.7 These information systems have multiple clinical and public health functions, all of which assist in ensuring appropriate vaccination to reduce risk for VPD.

Based on the results of a systematic review, the Community Preventive Services Task Force (Task Force) recently recommended use of IIS as an effective intervention to increase vaccination rates.8 Subsequently, this systematic economic review was conducted to evaluate costs and benefits associated with implementing, operating, and participating with an IIS.

Economic evaluations of technologically based interventions are challenging because of the speed and complexity of technology change. This challenge is particularly applicable to IIS, as systems have undergone major advancements since inception in the 1970s. This review identified the following 5 factors in the dynamic nature of IIS that complicate economic evaluation: (1) evolution of system technology; (2) evolution of data exchange methodologies; (3) emergence of software vendors that support IIS; (4) continued enhancements to system functionality; and (5) shift in the scope of IIS. With these factors in mind, this economic evaluation was conducted and assessed, and the results were compared with criteria of modern systems and capabilities.

Back to Top | Article Outline

Costs and benefits of IIS

Historically, in the United States, funding for IIS development has been provided by federal, state, local governments, private foundations, and managed care organizations (MCOs).9,10 Implementing IIS requires upfront investment from these payers, whereas the benefits—both financial and nonfinancial—accrue to the payers, providers, patients, and the general population. For example, using an IIS to prevent a duplicative vaccination reduces costs to the patient (eg, money, time, and pain of vaccination) and also reduces costs to providers, public health officials, and the general population by allocating fewer public and private resources to administer vaccines.

Expected costs of implementing an IIS include system costs (costs to develop and operate the IIS, to populate data at the central IIS, hardware and software costs, and/or licensing fees) and the cost of exchanging immunization data. Tables 1 and 2, respectively, provide an overview of system costs for 2 types of IIS and data exchange costs. Data exchange costs accrue to both the provider and the IIS. Data exchange costs to the provider include labor costs to enter data manually into IIS or costs associated with linking an existing software system (eg, an electronic health record) to IIS. Data exchange costs to IIS include identifying, enrolling, and training providers on use of IIS and establishing electronic linkages with other information systems, which incurs both operational and technical costs.

TABLE 1

TABLE 1

TABLE 2

TABLE 2

Economic benefits of IIS may include improved clinical service for vaccine administration and reduced administrative burdens associated with a vaccine delivery system. The ability of IIS to produce consolidated vaccination histories and to forecast (typically algorithm-based) which vaccinations are due and when those doses are due for each individual served by the system can support immunization providers in improving vaccination coverage and reducing overimmunization. Administrative support functions for vaccine delivery provided by IIS include increased efficiency of provider reminder functions, generation of reminder/recall notices for patients, and assessment and feedback interventions for providers.7 Immunization information systems may also be used to track vaccine stock and assess immunization activity for a provider practice. Improved vaccine supply and management should result in less waste and more accurate vaccine inventory and less time required to create vaccination coverage and assessment reports for the provider practice and the population. Ultimately, IIS can lead to an increase in appropriate vaccinations, which results in reduced VPD, an associated reduction in morbidity and mortality, and an improvement in quality-adjusted life-years. A reduction in morbidity and mortality is quantified as an economic outcome by measuring averted health care costs and productivity loss averted.

Back to Top | Article Outline

Evidence Acquisition

A systematic review of economic evaluation studies is typically conducted for community-based interventions recommended by the Task Force. Methods used by the Guide to Community Preventive Services (Community Guide) in conducting systematic reviews of economic evaluations are described elsewhere.11,12 To be eligible for inclusion in this economic review, studies had to satisfy the intervention definition stipulated in the effectiveness review—confidential, population-based, computerized databases that record all vaccination doses administered by participating providers to people residing within a given geopolitical area. Immunization systems that include multiple providers in a geographic area and represent a majority of a population are considered population-based for the purposes of this assessment. Search of the economic literature also mirrored the effectiveness search period of January 1994 to March 20127 and combined economic-specific key words such as cost, cost-benefit, cost-effectiveness, and cost utility with the effectiveness search terms. In addition to the databases searched in the effectiveness review (ie, The Cochrane Library; MEDLINE; CINAHL; PsycINFO; ERIC; Sociological Abstracts; Web of Knowledge; EMBASE; and CAB International), EconLit, Social Sciences Citations Index, JSTOR, and Google were used. All monetary values were adjusted to 2011 US dollars, using the general Consumer Price Index from the Bureau of Labor Statistics (www.bls.gov/).

To take into account the scale of IIS and increase comparability between studies, the Community Guide Economics Team (the Team) calculated cost per child vaccinated by using other measures provided in each study if this average cost information was not provided.

Back to Top | Article Outline

Evidence Synthesis

The economic search resulted in 71 potentially relevant studies after title and abstract screening. After a review of the full text of these studies, 12 published articles9,13–23 and 2 government reports24,25 were included as the final body of evidence. All studies assessed IIS in a US setting and focused on pediatric immunizations. Nine of these studies provided information related to either system costs or data exchange costs, and 9 provided information on benefits; 4 studies provided an assessment of both benefits and costs.

Back to Top | Article Outline

System cost

Seven studies9,14,16,18,21,22,25 evaluated IIS system cost (see Table 3), which includes costs of developing the system architecture, software, hardware, hardware configuration, populating the database, and training. Operation costs are the costs of maintaining the system and managing records. Three9,14,21 studies combined the cost of bringing providers online—equipment, data entry costs of the provider, and/or training costs—with development cost. Unfortunately, the studies did not stratify costs to distinguish systems cost from data exchange cost. Total costs ranged from $205 077 to $108 million, annual cost per child ranged from $5.40 to $60.82, and cost per vaccination record ranged from $0.11 to $12.88. Variability in costs might be attributable to the scale of the IIS and the target population size. Three studies16,18,21 evaluated a city IIS, whereas 2 studies14,22 evaluated a combination of city/state (or county) IIS. One study25 solely evaluated a state IIS, whereas an additional study9 focused on a national projection.

TABLE 3

TABLE 3

Back to Top | Article Outline

Data exchange costs

Two19,20 studies reported data exchange costs associated with IIS (see Table 4). One study20 reported operational costs from the provider perspective of reporting vaccination information to a central registry, and another study19 estimated costs for a MCO to link to an existing IIS to improve reporting for the Healthcare Effectiveness and Data Information Set, quality measurement, and the physician incentive program. Data exchange costs ranged from $8395 to $33 459. No evaluated studies reported explicit data exchange costs accrued by IIS. As mentioned previously, these costs were included in the development costs from 3 studies.9,14,21

TABLE 4

TABLE 4

Back to Top | Article Outline

Benefits

Nine studies9,13,15,17–19,23–25 focused on savings that could accrue from using an IIS. Most studies focused on administrative efficiency of an IIS in contrast to manually performed vaccination-related activities (eg, pulling records [health care and education system], contacting previous providers for vaccination histories, and generating immunization-related reports). Four studies9,15,23,25 considered reduction in costs that would result from decreased overvaccination. Of these studies, 215,23 estimated benefits of IIS during a public health emergency, a rare but critical event. To estimate potential savings from reduced vaccination duplication, both studies assumed that every record identified retrospectively through the Louisiana IIS, LINKS (Louisiana Immunization Network for Kids Statewide), represented savings from avoiding revaccination of children displaced during Hurricane Katrina. This assumption—that every child with a record in the LINKS would have been revaccinated in the absence of the system—likely leads to an overestimate of administrative savings; however, the estimate does not account for unnecessary pain and the inconvenience of reimmunization, nor for the costs associated with lost work time and school absenteeism because of vaccination appointments.15 The other studies9,25 modeled all potential benefits from reduced overvaccination (in addition to other benefits) on the basis of assumptions designated by the authors.

Benefits evaluated in these studies were mostly specific to savings associated with administrative efficiency and decreased overvaccination and thus provide a limited picture of benefits that might be realized with IIS. No studies evaluated economic benefits that result from reduced morbidity and mortality from VPD. Benefit outcomes evaluated in the included studies are presented in Table 5.

TABLE 5

TABLE 5

Back to Top | Article Outline

Cost-benefit

Four studies9,18,19,25 provided an assessment of benefits and program costs, each focusing on a different perspective (national, local city/state, or health care system). One study9 modeled cost and benefits of a nationwide IIS and indicated a benefit-cost ratio of 3.5:1. A state-level study25 modeled anticipated yearly benefits; compared with annual IIS costs to the state, the benefit-cost ratio was 1.59:1. At the city level,18 one evaluation found that using an IIS compared with manually performing immunization activities resulted in net savings of $36 815. Among health care systems, a MCO19 estimated a return of $8 for every $1 spent after linking to a state IIS for electronic data reporting versus manually retrieving claims data.

Back to Top | Article Outline

Conclusions

Cost studies in this review provide information on costs to implement and operate the system and associated costs to participate and exchange information with an IIS. Studies with benefit information focused on administrative efficiency of clinical vaccination activities and savings resulting from decreased overvaccination. A major challenge to evaluating a technology-based intervention is the evolution that comes with technology improvements and advancements. To determine whether cost and benefit estimates from the evaluation provide insight into the economic efficiency of present-day IIS, the IISs evaluated in the included body of evidence are compared with standards of more recent systems (discussed in the next section). To assess whether the evidence reflects mechanisms, capabilities, and scale of present-day IIS, the Team considered system technology, use of vendor support, data exchange methods, system functionality, and scope of IIS.

Back to Top | Article Outline

Relevance of findings

Six of the studies that assessed IIS systems cost date from 1997 to 2002 and thus reflect an IIS created at least 10 years before the evidence was gathered in 2012. As would be expected with a technology-based intervention, IIS system technology has evolved over time. When IISs were first developed in the United States, these systems primarily used mainframe technology with limited or no network connectivity. As the technology infrastructure improved, most IISs transitioned to client-server computer systems that enabled more automated data exchange by using a networked approach. Costs associated at this stage involved purchasing servers to support the IIS and information technology staff to maintain server functionality. Although many IISs continue to use client-server technologies, some are exploring use of cloud-based technology, which distributes computing infrastructure outside the organization, with more advanced networking. Costs associated with cloud-based technology are expected to be less than those associated with client-server technologies because of economies of scale and efficiency improvements and requirement of lower up-front costs.26,27 Unfortunately, only 1 study16 discussed details of the technological setup of the IIS. Two types of system technology were represented in the single study—mainframe architecture and client-server architecture. No studies reported costs associated with cloud-based technology.

Another evolution in IIS development has been the emergence of software vendors that support IIS. Most IISs were originally developed, coded, quality tested, and maintained by in-house information technology staff. Several major vendors now support IIS implementers. In 2013, 77% (43/56) of immunization programs in the United States were supported by IIS vendors (L. P., unpublished data, 2014). Because no studies included in this review evaluated costs of vendor-supported IIS, expenses associated with use of this technology compared with original in-house systems were not clear. However, it can be postulated that vendor-supported systems incur lower development and maintenance costs, particularly for IIS that are less customized. Vendor-supported software allows implementers to take advantage of investments already made in software development by the vendor, as well as predecessors who customized components of the system, rather than developing a system in-house. Correspondingly, some states have also shared costs in developing and implementing an IIS. In this review, many IISs in the studies represent the early adopters of IIS and thus are expected to have higher costs for using a new technology with no precedent system.

Third, methods for immunization data exchange have also changed over time. When IISs were first developed, immunization information was sent from the IIS on diskettes to vaccination providers who then loaded that information onto locally supported computer systems, updated that information with new vaccinations, and returned diskettes to the IIS. Other users relied primarily on paper reporting, which required that vaccination providers record immunization information on standard forms and mail or fax that information to the IIS. As the Internet became widely used, IIS transitioned to Web-based reporting systems so that providers could log on to a secure IIS Web site, then retrieve and submit immunization information in real time. As computers and electronic systems became commonplace in provider practices (eg, electronic billing systems and electronic health records), many providers and IIS developed capacity to exchange immunization information electronically between these systems. Electronic data exchange originally relied on batch reporting, in which data were queued up and sent to receiving systems at scheduled times. However, advances in system technologies, combined with use of Health Level 7 (HL7) standards, a nationally recognized standard for electronic data exchange between systems storing health data, have further supported the ability of IIS and provider-based systems to exchange data bidirectionally in real time. Costs associated with electronic data exchange include cost of information technology staff to maintain and support electronic data exchange for both the provider and the IIS. Only 2 studies provided information on data exchange costs.19,20 Of these studies, 1 measured costs associated with manual data entry performed by clinic personnel versus a billing or patient management system.20 However, it is unclear whether that study accounted for the technician costs to link the system to the IIS; the study reported only time and equipment costs. The other study is from the perspective of an MCO and included cost of linking existing internal electronic systems to IIS.19 The MCO did not directly input data into the IIS; consequently, the estimate might be applicable to a provider with an electronic health record system.

Fourth, rapid advances in computing and technology have produced numerous benefits by enabling new functions and improving existing functions. Generally speaking, IISs were originally developed to primarily consolidate vaccination histories across multiple providers to provide clinical decision support, conduct reminder/recall, and monitor vaccination coverage. However, in recent years, the functions and features of IIS have expanded to support the diverse needs of multiple immunization stakeholders, such as vaccine ordering and inventory management functions and emergency preparedness support, and have improved reporting functions to better address geographic pockets of needs (L. P., unpublished data, 2014). Most benefits in this body of evidence focused on reduction of labor and time costs that accrued from using computerized versus manual systems. Although 4 studies evaluated potential savings that would result from reduced duplicative immunizations because of better tracking and consolidated access of patient immunization records, the modeled/partially modeled findings have limitations because of their assumptions and hypothetical projections.

The addition and expansion of IIS functionality is associated both with costs, which are unclear, and benefits. This review provides only a small glimpse into the economic benefits that result from implementation of IIS. The absence of reporting many of the expected benefits does result in an underestimation; however, it is important to note that the benefits evaluated in this review are relevant to current IIS systems and can be viewed as the minimum benefits that can be expected.

Finally, IIS economies of scale have changed. Many of the IISs in this review focused on citywide systems. Today, most of these city-based systems have been consolidated into a state-based IIS that serves a larger population, helping to reduce costs associated with developing, maintaining, and connecting multiple smaller-scale systems. Immunization information systems have also expanded their original focus of childhood vaccinations to include adolescents and adults. As more IISs incorporate vaccine administration data for adults, either through adding adult patients to the system or as individuals already enrolled in the IIS age into adulthood, the capabilities of IIS will become more universally applicable to individuals of all ages. Ninety percent (53/56) of immunization programs are supported by an IIS that serves the life span and have become universally applicable to individuals of all ages (L. P., unpublished survey, 2014). The expansion of populations served increases the volume of data received, processed, and reported. Unfortunately, costs associated with enhancing IIS to be able to support this increase in volume were not available. With these limitations in mind, Table 6 provides a summary of why the evidence provided in the included studies might be less relevant in determining costs of a present-day IIS.

TABLE 6

TABLE 6

Back to Top | Article Outline

Summary of findings

Based on the limitations discussed in this article, evidence in this review might not accurately represent present-day IIS; however, it is unlikely that the ratio of economic benefits to costs is any less favorable. As IIS functionality has improved, benefits are expected to be greater than what is captured in this review. With multiple factors influencing cost—potentially lowering costs through advancements in technology, introduction of vendor-supported IIS, and increase in scale, and potentially raising costs through improved functionality—it is unclear how present-day costs would compare with costs reflected in the studies. However, it is unlikely that costs increased at the same magnitude as benefits, which were underestimated in this review; costs might also potentially be lower. Therefore, it is likely that more up-to-date estimates of costs and benefits would support the findings of cost savings in this review. In addition, more research is needed to update and address the limitations in available evidence and enable an assessment of economic costs and benefits of a present-day IIS.

Back to Top | Article Outline

Evidence Gaps

More economic data are needed on the costs of implementing present-day IIS, with information detailing whether the system is developed in-house or vendor supported. Details of system technology would be helpful in comparing each type of technology with its associated costs. This body of evidence primarily captured the economic benefit of reducing reporting burden and time spent locating records. Additional benefits (improved efficiencies and decreased time associated with conducting provider reminder/recall functions, provider assessment and feedback efforts, improved vaccine supply and management, and reduced morbidity and mortality) need to be monetized to fully capture the economic returns that accrue from using an IIS.

Lastly, the number of direct linkages between IIS and electronic health records (EHRs) are increasing in part due to the Centers for Medicare & Medicaid Services Meaningful Use initiative that provides financial incentives to eligible health care providers that acquire and demonstrate meaningful use of certified EHR products, which includes the exchange of data with IIS.28 As more providers create direct linkage from EHRs to the central IIS, more information and details are needed about the costs of this type of connection.

Back to Top | Article Outline

Discussion

This review provides an initial assessment of costs and benefits of implementing an IIS. As systems have evolved over time through technological innovation, in coordination with implementer input, competition, and experience, new approaches have been developed to build IISs that are more streamlined, operate faster, and have more capabilities. All included studies provide insight into the types of costs that might be incurred for implementers interested in building an IIS; however, dollar figures are less relevant because the systems evaluated are outdated.

The primary difficulty in assessing economic evidence was the rapid change in technology costs. As health information systems are increasingly adopted across public health settings, understanding upfront and ongoing costs, alternative costs of software, and associated benefits is key to determining the value of interventions. However, with continued rapid advancement of technology, challenges faced in this review will likely be a problem for other technology-based reviews. One approach might be to restrict the search inclusion criteria to exclude older technology or categorize the relevance of studies by comparing with more recent technology.

Reporting costs faced by vaccine providers who participate in IIS were also reviewed. Advancements in technology now allow providers to share immunization data through a direct link to the central IIS. However, cost barriers exist for providers to create the direct linkage, and significant administrative efforts and trainings might be required to re-engineer and align the providers' immunization practices to take full advantages of functionality efficiencies of IIS.16

Another public health goal is to integrate a vaccine-ordering module within an IIS that interfaces with the Centers for Disease Control and Prevention's Vaccine Tracking System for ordering and managing vaccine distribution.29 Linking systems would streamline the process and potentially save money by reducing errors and waste and ensure that appropriate supplies of vaccines are distributed to providers.

Back to Top | Article Outline

REFERENCES

1. Roush SW, Murphy TV. Vaccine-Preventable Disease Table Working Group. Historical comparisons of morbidity and mortality for vaccine-preventable diseases in the United States. JAMA. 2007;298(18):2155–2163.
2. Centers for Disease Control and Prevention. Ten great public health achievements—United States, 2001-2010. MMWR Morb Mortal Wkly Rep. 2011;60(19):619–623.
3. Zhou F, Shefer A, Wenger J, et al. Economic evaluation of the routine childhood immunization program in the United States, 2009. Pediatrics. 2014;133(4):577–585.
4. Fedson DS, Fulginiti VA, Bloom BR, et al. Adult immunization. JAMA. 1994;272(14):1133–1137.
5. Briss PA, Rodewald LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1S):97.
6. Orenstein WA, Papania MJ, Wharton ME. Measles elimination in the United States. J Infect Dis. 2004;189(1S):S1–S3.
7. Groom H, Hopkins D, Pabst L. Immunization information systems to increase vaccination rates: a Community Guide systematic review. J Public Health Manag Pract. 2014;227–248.
8. Community Preventive Services Task Force. Use of immunization information systems to increase vaccination rates: a recommendation of the Community Preventive Services Task Force. J Public Health Manag Pract. 2014;249–252.
9. Horne PR, Saarlas KN, Hinman AR. Costs of immunization registries: experiences from the All Kids Count II projects. Am J Prev Med. 2000;19(2):94–98.
10. Linkins RW, Salmon DA, Omer SB, Pan WKY, Stokley S, Halsey NA. Support for immunization registries among parents of vaccinated and unvaccinated school-aged children: a case control study. BMC Public Health 2006;6(1):236.
11. Carande-Kulis VG, Maciosek MV, Briss PA, et al. Methods for systematic reviews of economic evaluations for the guide to community preventive services. Am J Prev Med. 2000;18(1S):75–91.
12. Zaza S, Carande-Kulis VG, Sleet DA, et al. Methods for conducting systematic reviews of the evidence of effectiveness and economic efficiency of interventions to reduce injuries to motor vehicle occupants. Am J Prev Med. 2001;21(4S):23–30.
13. Bartlett DL. Cost savings associated with using immunization information systems for vaccines for children administrative tasks. J Public Health Manag Pract. 2007;13(6):559.
14. Bartlett DL, Molinari NAM, Ortega-Sanchez IR, Urquhart GA. Economics of immunization information systems in the United States: assessing costs and efficiency. Cost Eff Resour Alloc. 2006;4(1):15.
15. Boom JA, Dragsbaek AC, Nelson CS. The success of an immunization information system in the wake of Hurricane Katrina. Pediatrics. 2007;119(6):1213–1217.
16. Fontanesi JM, Flesher DS Jr, De Guire M, Lieberthal A, Holcomb K. The cost of doing business: cost structure of electronic immunization registries. Health Serv Res. 2002;37(5):1291–1307.
17. Glazner JE, Beaty BL, Pearson KA, Lowery NE, Berman S. Using an immunization registry: effect on practice costs and time. Ambul Pediatr. 2004;4(1):34–40.
18. McKenna VB, Sager A, Gunn JE, Tormey P, Barry MA. Immunization registries: costs and savings. Public Health Rep. 2002;117(4):386.
19. O'Connor AC, Layton CM, Osbeck TJ, Hoyle TM, Rasulnia B. Health plan use of immunization information systems for quality measurement. Health. 2010;16:217–224.
20. Rask KJ, Wells KJ, Kohler SA, Rust CT, Cangialose CB. The cost to providers of participating in an immunization registry. Am J Prev Med. 2000;19(2):99–103.
21. Rask KJ, Wells KJ, Kohler SA, Rust CT, Cangialose CB. Measuring immunization registry costs: promises and pitfalls. Am J Prev Med. 2000;18(3):262–267.
22. Slifkin RT, Freeman VA, Biddle AK. Costs of developing childhood immunization registries: case studies from four All Kids Count Projects. J Public Health Manag Pract. 1999;5:67–81.
23. Urquhart GA, Williams W, Tobias J, Welch FJ. Immunization information systems use during a public health emergency in the United States. J Public Health Manag Pract. 2007;13(5):481–485.
24. Florida Bureau of Immunization. Florida SHOTS time savings study results. http://www.flshots.com/_pdfs/TimeSavingsStudyFlyer.pdf. Published 2009. Accessed August 2, 2010.
25. Virginia Department of Health. 12 VAC 5-115- Virginia immunization information system economic impact analysis. Virginia Department of Planning and Budget. Va Regist Regul. 2012;29(6). http://register.dls.virginia.gov/details.aspx?id=3286. Accessed February 3, 2013.
26. Marks G. Do you replace your server or go to the cloud? The answer may surprise you. www.forbes.com/sites/quickerbettertech/2013/04/29/do-you-replace-your-server-or-go-to-the-cloud-the-answer-may-surprise-you/. Publi-shed 2013. Accessed January 5, 2014.
27. Potter K. Cloud Computing: Economic, Financial and Service Impact on IT Planning Assumptions. Stamford, CT: Gartner Inc; 2013.
28. Centers for Medicare and Medicaid Services. Meaningful use. http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Meaningful_Use.html. Accessed May 5, 2014.
29. Centers for Disease Control and Prevention. PPHF 2012—Prevention and Public Health Fund (Affordable Care Act)—Capacity building assistance to strengthen public health immunization infrastructure and performance financed in part by 2012 Prevention and Public Health Funds. https://www.cfda.gov/?s=program&mode=form&tab=step1&id=256e273b6e05d96d1b298d67f28a2d43. Accessed February 20, 2014.
Keywords:

benefit; Community Guide; Community Preventive Services Task Force; cost; cost-effectiveness; economics; immunization; immunization information system; systematic review

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.