Immunization Information Systems to Increase Vaccination Rates: A Community Guide Systematic Review

Groom, Holly MPH; Hopkins, David P. MD, MPH; Pabst, Laura J. MPH; Murphy Morgan, Jennifer MSPH; Patel, Mona MPH; Calonge, Ned MD; Coyle, Rebecca MPH; Dombkowski, Kevin DrPH, MS; Groom, Amy V. MPH; Kurilo, Mary Beth MPH, MSW; Rasulnia, Bobby PhD, MPH, MPA; Shefer, Abigail MD; Town, Cecile MPH; Wortley, Pascale M. MD, MPH; Zucker, Jane MD, MS; the Community Preventive Services Task Force

Journal of Public Health Management & Practice: May/June 2015 - Volume 21 - Issue 3 - p 227–248
doi: 10.1097/PHH.0000000000000069
Original Articles

Context: Immunizations are the most effective way to reduce incidence of vaccine-preventable diseases. Immunization information systems (IISs) are confidential, population-based, computerized databases that record all vaccination doses administered by participating providers to people residing within a given geopolitical area. They facilitate consolidation of vaccination histories for use by health care providers in determining appropriate client vaccinations. Immunization information systems also provide aggregate data on immunizations for use in monitoring coverage and program operations and to guide public health action.

Evidence Acquisition: Methods for conducting systematic reviews for the Guide to Community Preventive Services were used to assess the effectiveness of IISs. Reviewed evidence examined changes in vaccination rates in client populations or described expanded IIS capabilities related to improving vaccinations. The literature search identified 108 published articles and 132 conference abstracts describing or evaluating the use of IISs in different assessment categories.

Evidence Synthesis: Studies described or evaluated IIS capabilities to (1) create or support effective interventions to increase vaccination rates, such as client reminder and recall, provider assessment and feedback, and provider reminders; (2) determine client vaccination status to inform decisions by clinicians, health care systems, and schools; (3) guide public health responses to outbreaks of vaccine-preventable disease; (4) inform assessments of vaccination coverage, missed vaccination opportunities, invalid dose administration, and disparities; and (5) facilitate vaccine management and accountability.

Conclusions: Findings from 240 articles and abstracts demonstrate IIS capabilities and actions in increasing vaccination rates with the goal of reducing vaccine-preventable disease.

This article provides findings from 240 articles and abstracts demonstrating immunization information system capabilities and actions in increasing vaccination rates with the goal of reducing vaccine-preventable disease.

Community Guide Branch, Division of Epidemiology, Analysis, and Library Services (Dr Hopkins and Mss Murphy Morgan and Patel), Immunization Services Division, National Center for Immunizations and Respiratory Diseases (Mss Groom, Pabst, Shefer, and Town and Dr Zucker), Office for State, Tribal, Local, and Territorial Support (Dr Rasulnia), and Division of HIV/AIDS Prevention (Dr Wortley), Centers for Disease Control and Prevention, Atlanta, Georgia; The Colorado Trust, Denver, Colorado (Dr Calonge); American Immunization Registry Association (AIRA), Washington, District of Columbia (Ms Coyle); Child Health Evaluation and Research (CHEAR) Unit, University of Michigan, Ann Arbor, Michigan (Dr Dombkowski); and Oregon Immunization Program, Portland, Oregon (Ms Kurilo). Ms Groom is now with Center for Health Research, Kaiser Permanente Northwest, Portland, Oregon.

Correspondence: Holly Groom, MPH, Center for Health Research, Kaiser Permanente Northwest, 3800 N Interstate Ave, Portland, OR 97227 (Holly.c.groom@kpchr.org).

Names and affıliations of Task Force members are available at www.thecommunityguide.org/about/task-force-members.html.

We gratefully acknowledge the team of consultants, referred to as the “team,” who helped us form the framework and research questions, and establish criteria for included studies for this review. We are also grateful to the CDC Immunization Information System Support Branch for the subject matter expertise they provided, and to Gary Edgar for his support throughout the review process. In addition, the authors thank Dr Robert Menzies, from the National Centre for Immunisation Research and Surveillance of Vaccine Preventable Diseases (NCIRS) at the University of Sydney, for his input on the sections pertaining to the Australian Childhood Immunisation Register.

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

The authors declare no conflicts of interest.

This is an open-access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License, where it is permissible to download and share the work provided it is properly cited. The work cannot be changed in any way or used commercially.

Article Outline
Back to Top | Article Outline

Context

Immunizations are among the top 10 great public health achievements of the 20th century for their success in realizing substantial declines in cases, hospitalizations, deaths, and health care costs associated with vaccine-preventable diseases.1,2 In the United States, population-based vaccine delivery requires collaboration of public and private health care providers with local, state, and federal governments and public health agencies.3 Efforts to achieve and maintain high levels of vaccine coverage in the population require implementation and coordination of a wide range of public policy, health system, and community-based interventions.

Ensuring well-coordinated activities to foster high immunization coverage levels is dependent on the availability of timely, accurate, and complete information pertaining to vaccinations received by members of a population.4 Immunization information systems (IISs) provide a potentially powerful tool to allow collaboration between vaccination providers and public health agencies and for coordination of population-based interventions. They are confidential, population-based, computerized databases that record all vaccination doses administered by participating providers to people residing within a given geopolitical area.5 At the point of clinical care, an IIS can provide consolidated immunization histories to assist vaccination providers in determining appropriate client vaccinations. At the population level, an IIS provides aggregate data on vaccinations for use in assessments of coverage and program operations and in guiding public health action to improve vaccination rates.

The development of IISs in the United States, which began in the 1970s, has been spurred by funding from federal, state, and private institutions and aided by policy recommendations and program standards disseminated by the National Vaccine Advisory Committee since 1999.6 The Centers for Disease Control and Prevention (CDC) makes annual revisions to the IIS Functional Standards based on input from a variety of technical experts, which are disseminated to IIS partners, nationally (http://www.cdc.gov/vaccines/programs/iis/func-stds.html). In 2011, 55 of 56 US state and city immunization grantees used an IIS.5

Since 2001 in the United States, a subset of IISs with higher levels of child participation and provider site reporting has participated in a collaboration with the CDC as part of the “IIS Sentinel Site Project” both to conduct population-based vaccination studies using IIS data and to implement systematic data quality improvement activities (http://www.cdc.gov/vaccines/programs/iis/activities/sentinel-sites.html). From 2001 to 2003, 8 IISs (Arizona, District of Columbia, Michigan, New York City, Oklahoma, Oregon, San Antonio, and Utah) participated in a pilot project. Sentinel Site grant funds were awarded to 6 sites (Arizona, District of Columbia, Michigan, Minnesota, Montana, and Oregon) for the 2004-2007 project period, 7 states (Arizona, Colorado, Michigan, Minnesota, North Dakota, Oregon, and Wisconsin) and 1 city (New York City) for the 2008-2012 project period, and 6 sites (Michigan, Minnesota, North Dakota, New York City, Oregon, and Wisconsin) for the 2013-2017 project period. Sentinel Site grant funds were awarded on the basis of established data quality measures including:

* 85% or more of children younger than 19 years have 2 or more immunization records in the IIS;

* 85% or more of vaccine provider sites are enrolled in the IIS; and

* 70% or more of vaccine doses administered in the Sentinel Site area are submitted to and processed by the IIS within 30 days of administration.

A number of countries, including Australia, Canada, Denmark, Italy, the Netherlands, New Zealand, Norway, Scotland, Sweden, and the United Kingdom, in addition to the United States, have implemented stand-alone IISs or incorporated vaccination data into a more comprehensive medical record system. One stand-alone IIS with capabilities and activities similar to US systems is the Australian Childhood Immu-nisation Register (ACIR) (http://www.humanservices.gov.au/customer/services/medicare/australian-childhood-immunisation-register). The ACIR is a national register administered by Medicare Australia that covers children younger than 7 years, with participation rates at 99%.7

Considerable costs are associated with IISs, because the systems require time and effort of participating vaccination providers, as well as ongoing staffing and financial support from state and federal partners. As both programs and providers work to balance the utility and value of IIS with these infrastructure and participation costs, an assessment of IIS capabilities and effectiveness in improving immunization rates and preventing vaccine-preventable disease is especially important.

Back to Top | Article Outline

Evidence Acquisition

Community Guide methods (http://www.thecommunityguide.org/about/methods.html) were used to conduct this systematic review to determine the effectiveness of IIS in increasing vaccination rates, reducing vaccine-preventable disease, or enhancing vaccination program capabilities.8 Briefly, a coordination team* (“the team”) was constituted, including subject matter experts from various agencies, organizations, and academic institutions, together with qualifıed systematic reviewers. For this review, the team included CDC staff from Immunization Information Systems Support Branch in the National Center for Immunization and Respiratory Diseases, CDC field staff, immunization staff working at state health departments, and individuals within academic institutions and health care systems. The team worked under the oversight of the Community Preventive Services Task Force (Task Force), an independent, nonfederal, unpaid panel of public health experts.

For each Community Guide systematic review, the team (i) establishes a conceptual approach to assess the evidence of effectiveness of a given intervention in improving health at the population level; (ii) develops an analytic framework depicting interrelationships among interventions, populations, and outcomes; (iii) systematically searches for and retrieves evidence; (iv) evaluates the strengths and limitations of the body of evidence; (v) summarizes the evidence on effectiveness and calculates summary measures, if possible, for health-related outcomes; (vi) concludes on public health benefits from the intervention of interest; (vii) summarizes information on applicability (ie, the extent to which conclusions are generalizable to diverse population segments and settings in the US context), economic impact, additional benefits, potential harms, and considerations for implementation; and (viii) identifies important evidence gaps.

The Task Force considers results from the review process taking into account the population-level importance of the overall body of evidence, effect estimates, and supporting information to reach consensus conclusions on recommendations for public health practice and policy. Task Force options include recommending the use of an intervention (practice or policy), recommending against use, or finding that evidence available is insufficient to determine effectiveness.

Back to Top | Article Outline

Conceptual approach and analytic framework

The analytic framework (Figure) postulates the impact of IIS on a wide range of vaccination provider and public health outcomes. Based on timely and complete reporting from vaccination providers, IISs maintain consolidated immunization histories, which can be retrieved in clinical settings to determine appropriate vaccinations for clients. IISs have the ability to support providers through provider reminder functions (also referred to as clinical decision support systems for immunizations, or CDSi), generation of reminder/recall notices for clients, and provision of data to support assessment and feedback interventions efficiently. School systems can use IIS records to determine whether students meet entry requirements. At the population level, IISs can be used to assess local vaccination coverage levels or to assist during disease outbreaks, public health emergencies, or vaccine shortage situations to identify individuals in need of vaccination. In addition, IISs can be used in vaccine management and accountability, in tracking of vaccine safety issues, and in the evaluation of vaccine effectiveness.

In this review, IISs were also considered for their ability to support additional vaccination-related interventions and to enhance or expand vaccination program capabilities. These included 3 interventions previously reviewed and recommended by the Task Force on the basis of evidence of effectiveness in increasing vaccination rates:

* Client reminder and recall notices, disseminated either through mail or by phone, used to remind members of a target population that vaccinations are due (reminders) or late (recall).

* Provider assessment and feedback, which involves, retrospectively, evaluating the performance of providers in delivering 1 or more vaccinations to a client population and giving providers feedback on their performance.

* Provider reminder systems (or CDSi), used to prompt providers about due or past due vaccinations for a client being seen.

Each of these interventions was found to be effective as a stand-alone intervention (http://www.thecommunityguide.org/vaccines/index.html); this review, however, assessed effectiveness of implementing each intervention through the use of an IIS or a population-based vaccination database.

Back to Top | Article Outline

Research questions

The review team generated the following research questions for this review:

1. How effective are IISs in creating or supporting interventions to increase vaccination rates or reduce vaccine-preventable disease?

2. Do IISs effectively support public health efforts to determine client vaccination status to inform decisions by clinicians, health care systems, and schools?

3. Do IISs effectively support public health responses to outbreaks of vaccine-preventable disease?

4. Do IISs inform assessments of vaccination coverage, missed vaccination opportunities, invalid dose administration, and disparities?

5. Do IISs facilitate vaccine management and accountability?

Back to Top | Article Outline

Search for evidence

The following databases were searched for this review: The Cochrane Library; MEDLINE; CINAHL; PsycINFO; ERIC; Sociological Abstracts; Web of Knowledge; EMBASE; and CAB International. The search period was January 1994 to April 2011. Details of the search strategy are available (see the Appendix). Reference lists of retrieved articles and reviews were also searched, and subject matter experts were consulted to identify studies that might have been missed.

The team also decided to include an assessment of unpublished US literature, in the form of conference abstracts, in order to capture more recent information about how IISs are being used to support vaccination-related interventions and expand vaccination program capabilities in the United States. Abstracts presented at 4 selected conferences (Immunization Registry Conference; National Immunization Conference; Pediatric Academic Society; Vaccine University) in the United States in the period from January 2002 to April 2011 were screened and relevant evidence was included in this review. Because abstracts, in general, provided only summaries of information typically evaluated as part of the Community Guide assessment of study quality of execution, the team decided not to conduct quality assessments of included studies in this review in favor of an overall assessment of limitations in the included evidence.

Back to Top | Article Outline

Inclusion criteria

To qualify as a candidate for inclusion in this review, a study had to:

* Evaluate the effectiveness of an IIS or population-based vaccination database, or the effectiveness of an intervention generated from an IIS, or describe an IIS capability linked to increasing vaccination rates or reducing vaccine-preventable disease.

* Be conducted in a high-income country,* be a primary research publication or US conference abstract, and be published in English.

* Report 1 or more quantitative outcomes (changes in vaccination rates, coverage, or uptake; reductions in missed opportunities) or describe system capabilities linked to increasing vaccinations or reducing vaccine-preventable disease.

Included studies focused on IISs or other population-based vaccination databases intended to consolidate immunization data from all participating providers in a geographic area. This review did not include studies evaluating encounter-based data systems and electronic health records (EHRs) as primary data sources based on team and Task Force concerns about applicability of that evidence to current IISs.

Back to Top | Article Outline

Assessing and summarizing evidence on effectiveness

Each study that met the inclusion criteria was read by 2 reviewers who used standardized criteria to abstract study details and assess the suitability of study design.8 Uncertainties and disagreements between reviewers were reconciled by consensus among review team members. Studies that collected data on exposed and comparison populations prospectively were classified as having the greatest design suitability. Those that collected data retrospectively or lacked a comparison group, but conducted multiple pre- and postmeasurements on their study population(s), were rated as having moderate design suitability. Studies with least suitable designs were cross-sectional studies and those that involved only a single pre- or postmeasurement in the intervention population, as well as studies providing descriptions of program capabilities or the use of an IIS for vaccination-related decisions by clinicians, health systems, schools, and public health agencies.

Studies meeting inclusion criteria were categorized into appropriate evidence categories, described later, for abstraction and assessment. Articles providing unique effect estimates or descriptions of program capabilities were considered as distinct studies within this review, even when the information came from an evaluation of the same immunization information system or vaccination database. Most study findings were organized and summarized in a narrative format. In a few categories, studies provided quantitative measurements of change in vaccination rates, and for this evidence, effect estimates were calculated as absolute percentage point change and summary estimates included the median effect estimate and the interquartile interval.

Back to Top | Article Outline

Economic Evaluation

An independent Community Guide review of the economic evidence regarding IISs was carried out. Findings are described elsewhere (Patel et al, in press, 2014).

Back to Top | Article Outline

Evidence Synthesis

Body of evidence of included studies

The review included 108 published articles and 132 conference abstracts for a total of 240 studies in 9 assessment categories:

1. Overall effects on vaccination rates;

2. Assessments of IIS-supported interventions to increase vaccination rates;

3. Use of IIS in vaccination-related decisions by clinicians, schools, and health care systems;

4. IIS-supported outbreak and public health emergency responses;

5. IIS for vaccine ordering, inventory, and accountability;

6. IIS to assess vaccine safety and effectiveness;

7. IIS as a tool for public health decision support;

8. IIS integrated with other child health information systems; and

9. IIS as contributing to change in documentation.

Sixteen articles had study designs of greatest suitability, and the remaining 224 studies had moderate (n = 36) and least (n = 188) suitable designs. Most studies (n = 209) evaluated systems in the United States, 26 studies examined the national system in Australia, and 5 studies evaluated other national systems. Of the US studies, 108 articles (52%) described actions and capabilities of systems within the IIS Sentinel Site program (http://www.cdc.gov/vaccines/programs/iis/activities/sentinel-sites.html).

Back to Top | Article Outline

Effectiveness

Overall effects on vaccination rates

Two studies, both of least suitable designs, evaluated the association of IISs with changes in vaccination rates.9,10 The first study documented changes in vaccination rates among children following implementation of the ACIR in 1996.9 Following the linkage of ACIR participation with parental and provider financial incentives in 1998 and inclusion of immunizations administered overseas in 2001, reported rates for full immunization among children aged 24 months in Australia increased from 64% in 1997 to 92.7% in 2007. The second article, a cross-sectional study conducted in the United States, evaluated the association between practice use of an IIS and likelihood of children being up-to-date. Practices using an IIS between 2004 and 2006 did not have significantly higher coverage levels than those practices not using an IIS.10

Back to Top | Article Outline

Assessments of IIS-supported interventions to increase vaccination rates

The search identified 47 studies evaluating or describing at least 1 of 3 specific interventions to increase vaccinations supported directly by an IIS or population-based vaccination database: (1) client reminder and recall notices; (2) provider assessment and feedback activities; and (3) provider reminder systems.

Back to Top | Article Outline

Client reminder and recall

Thirty studies evaluated effectiveness of client reminder and recall notices generated from an IIS or vaccination database. All but one study were conducted in the United States. Thirteen studies, 9 of greatest suitability of design, were published in peer-reviewed journals11–23 and 17 studies, 5 studies24–28 of greatest suitability and 12 studies29–40 of moderate or least suitable design, were presented as conference abstracts.

Thirteen studies12–19,24,25,27–29 with 16 study arms provided measurements of change in vaccination rates with a median absolute percentage point improvement of 6 percentage points (pct pts; interquartile interval, 4-7 pct pts). The remaining 17 studies11,20–23,26,30–40 provided descriptive information about the use of IIS data to generate reminder/recall notices; 2 studies38,40 described using the IIS to recall children during a measles outbreak, and 3 studies21,35,37 recalled children who had received doses of a subpotent or recalled vaccine.

Back to Top | Article Outline

Provider assessment and feedback

A total of 15 studies, 14 US and 1 Australian, evaluated the use of IIS data to assess provider vaccination performance within an assessment and feedback intervention. Three studies, one of greatest suitability of design,15 were published in peer-reviewed journals,15,41,42 and 12 studies, 11 of least suitable design, were presented as conference abstracts.43–54

Five included studies15,44,46,49,53 provided measurements of change in vaccination rates and observed a median absolute percentage point increase of 9 pct pts (range, 5-15 pct pts). The 10 remaining studies41–43,45,47,48,50–52,54 provided descriptions of IIS-generated assessments, with 3 studies48,51,54 using IIS data to identify invalid doses administered to clients and 2 studies45,52 describing time saved by using an IIS compared with the previous practice of pulling and reviewing charts within the provider practice.

Back to Top | Article Outline

Provider reminders

Three US studies,55–57 all conference abstracts of least suitable design, described or evaluated the use of IIS-generated reminders to vaccination providers. One study56 evaluated effectiveness of a provider reminder system in increasing vaccination rates in a provider practice and found an absolute percentage point increase of 14.2 pct pts. Two studies55,56 described the use of Web-based IIS features as reminders to providers on vaccine availability and updated recommendations for administration of a pandemic influenza vaccine.

Back to Top | Article Outline

Use of IIS in vaccination-related decisions by clinicians, schools, and health care systems

The search identified 14 US studies58–71 evaluating or describing the use of IIS by vaccination providers, schools and day cares, or health care systems. All 14 studies were conference abstracts of least suitable design.

Back to Top | Article Outline

Use of IIS by vaccination providers

No studies identified in this review specifically evaluated or described the use of IIS by individual vaccination providers in making clinical decisions about client vaccinations. No evidence was identified in this review to inform an assessment of IIS access and frequency of use in clinical settings and to examine IIS use and clinical outcomes such as coverage rates and timeliness.

Back to Top | Article Outline

Use of IIS by schools and child care centers (day cares)

Ten US studies,58–60,63,64,66,68–71 all conference abstracts of least suitable design, described IIS interactions with schools and day cares. One abstract63 quantified school use of an IIS to print out certificates of immunization status. Eight abstracts58–60,64,66,68,69,71 described how inclusion of school and day care data in the IIS helped improve data completeness and accuracy, thereby becoming a more useful tool for assessing student immunization status. The remaining abstract70 described the use of the state IIS to determine whether the IIS included records of children with school-documented vaccine exemptions receiving the exempted vaccine.

Back to Top | Article Outline

Use of IIS by health care systems

Four studies,61,62,65,67 all of least suitable design, described the use of a state IIS by a health care system. One published study65 and 2 abstracts62,67 described the use of IIS data for quality measurements including HEDIS (Healthcare Effectiveness Data and Information Set) reporting; one of these studies adopted the IIS as the vaccination database. One abstract61 described the use of an IIS to screen and vaccinate health care workers on-site.

Back to Top | Article Outline

IIS-supported outbreak and public health emergency responses

Eighteen US studies, 4 of moderate suitability of design72–75 and 14 of least suitable design,38,40,55,76–86 evaluated or described the use of IIS in responses to outbreaks of vaccine-preventable disease or other public health emergencies. Three abstracts examined the use of IIS data to identify residents for targeted recall notices during outbreaks of measles38,40 and hepatitis A.79 For example, the New York City IIS was used to recall more than 7000 children younger than 5 years who were not fully vaccinated with the measles vaccine. Four conference abstracts72–74,80 and 1 published study75 described the use of IIS data in retrospective assessments of vaccination status or uptake following outbreaks of pertussis to determine whether providers complied with recommendations to use the accelerated vaccination schedule of the Advisory Committee on Immunization Practices. Seven conference abstracts56,77,78,81,82,84,86 described provider participation, client management, vaccine distribution, or reporting outcomes related to the use of IIS for vaccine allocation during the 2009 H1N1 influenza pandemic. Finally, 2 published studies76,85 and 1 abstract83 described the use of IIS data in vaccination decisions for clients displaced by Hurricane Katrina, quantifying substantial benefits attributable to reductions in unnecessary vaccinations. One assessment85 projected a total savings of $13.1 million in vaccine and administration fees by using IIS to retrieve vaccination histories and avoid overimmunization of individuals in areas affected by Hurricane Katrina.

Back to Top | Article Outline

IIS for vaccine ordering, inventory, and accountability

Fourteen studies,86–99 all of least suitable design, provided descriptions of the use of IIS in facilitating vaccine ordering, inventory monitoring, and vaccine-related accountability. In 10 studies, the focus was on inventory management of vaccines available through the federally funded Vaccines for Children (VFC) program (http://www.cdc.gov/vaccines/programs/vfc/index.html). One published study94 and 2 conference abstracts86,92 described the application of VFC vaccine tracking requirements for vaccination providers for H1N1 monovalent influenza vaccine doses during the 2009 H1N1 influenza pandemic. Six abstracts87,89,90,93,95,99 described modules for vaccine ordering within their IISs to improve vaccination provider accountability for VFC vaccine doses received. Four pilot studies described challenges96 and successes88,91,98 with the integration of the VFC-required Vaccine Tracking System (VTrckS) within existing IIS. Finally, 1 conference abstract97 described the use of IIS to facilitate billing for non-VFC doses to private insurers.

Back to Top | Article Outline

IIS to assess vaccine safety and effectiveness

Seventeen studies100–116 of least suitable design (12 in the United States) described the use of IIS in assessment of, or response to, vaccine safety issues or in evaluation of vaccine effectiveness. Four published studies103,111,113,115 and 1 conference abstract116 described activities related to vaccine safety. The abstract described implementation of a targeted recall of potentially contaminated vaccine in a large metropolitan area, where IIS data were used to identify both affected vaccination providers and patients.116 Three published studies described tracking of adverse events related to administration of the H1N1 monovalent influenza vaccine,113 the inadvertent administration of anthrax vaccine to pregnant women,111 the association between MMR (measles-mumps-rubella) and DTP (diphtheria and tetanus toxoids and pertussis) vaccines and adverse events,103 and the introduction of a new meningococcal vaccine in New Zealand.115

Vaccine effectiveness assessments, typically involving comparison of IIS immunization records with disease or illness case reports, were described in 12 studies.100–102,104–110,112,114 Two published studies examined general feasibility issues for US systems,106,110 but the remaining studies described evaluations of specific vaccines, including rotavirus vaccine (4 published studies and 1 abstract),101,102,104,112,114 pneumococcal conjugate vaccine (1 abstract),108 influenza vaccine in children (2 published studies),100,105 Haemophilus influenzae type b (Hib) vaccine (1 published study),107 and meningococcal vaccine (1 published study).109

Back to Top | Article Outline

IIS as a tool for public health decision support

The studies categorized in this section described IIS capabilities to provide information, such as vaccination rates or coverage trends, which might prompt additional public health intervention.

A total of 110 studies, 2 of greatest suitability14,117 and 108 of moderate or least suitable design,* were identified; 83 were conducted in the United States. Evidence included 57 published articles11,14,107,109,117–169 and 53 conference abstracts, which examined the use of IIS data in assessments of vaccination coverage for a wide range of vaccines and target populations. Within this body of evidence, most reports described IIS-based assessments of vaccination coverage in specific populations, including 32 published studies and 19 abstracts.§ Four studies154–157 published in the MMWR Morbidity and Mortality Weekly Report provided influenza vaccination coverage estimates using data from participating IIS Sentinel Sites within 3 months of the end of the influenza season. Prior to 2010, estimates of influenza vaccination coverage, assessed using data from the National Immunization Survey (NIS) or the National Health Interview Survey, were not published until the year after the most recent influenza season. Thirty-two studies (13 published, 19 abstracts) described IIS assessments in higher-risk individuals and populations. In 9 studies (1 published141), IIS records were used to assess potential disparities in vaccination coverage rates by neighborhood (“pockets of need”) or population characteristic. IIS records informed assessment of the uptake of a new vaccine recommendation in 10 published studies# and 6 abstracts.114,171,176,178,182,198 IIS-based assessments of age-appropriate and series completion rates were reported in 4 published studies127,130,144,162 and 5 abstracts.51,54,176,189,193 Missed opportunities were estimated, typically by comparing rates of simultaneous receipt of recommended vaccinations, in 4 studies.134,135,158,208 Four published studies11,120,168,169 evaluated vaccination coverage and provider compliance with interim recommendations during vaccine shortages. Finally, 1 abstract216 evaluated the extent of overimmunization among US children, as documented in 6 Sentinel Site IISs.

Back to Top | Article Outline

IIS integrated with other child health information systems

Eight US studies,217–224 all of least suitable design, discussed integration of additional child health data into an existing IIS. Two published articles220,221 described Michigan's experience with determining which systems were appropriate to integrate with the Michigan IIS, as well as steps taken to move legislation in support of allowing data to be used for public health purposes beyond immunizations. One published article and 2 abstracts describe the successful integration of immunization data with other data sources in New York City,223 New Jersey,222 and San Diego.217 The linkages described include blood lead levels, Early Hearing Detection and Intervention screening, and a body mass index calculation chart. Three other articles218,219,224 describe the importance of moving toward integrated child health information systems from the perspective of how it can improve health care providers' abilities to be more fully informed and effective in client management.

Back to Top | Article Outline

IIS as contributing to change in documentation

IIS data quality issues were evaluated in 15 published studies106,190,225–237 and 9 abstracts.238–246 Twenty-one studies compared the completeness of individual IIS vaccination records with medical records,* the NIS,232,239,246 or parental report.227,236,243 Thirteen of these studies were published between 2000 and 2006, and 2 abstracts239,245 were published between 2010 and 2011. In 8 studies,228–232,234,237,245 individual immunization assessments were more complete when the IIS or registry record was combined with medical record data; however, this statement was often paired with the conclusion that IIS data were incomplete and not adequate as stand-alone data sources for assessments of coverage in studies conducted prior to 2007. Three studies230,231,234 examined the impact on vaccination rates of aggregating private and public health department immunization data into a regional registry with a demonstration project that resulted in a registry comprising a vast network of practices in the Denver metropolitan area. In 3 studies,232,239,246 IIS coverage estimates were compared with estimates from the NIS. One study246 found that IIS provided comparable coverage to the NIS for birth-dose hepatitis B coverage. Two other studies232,239 found individual vaccination coverage to be higher in the NIS than in the IIS, and rates were highest when data sources were pooled.

Back to Top | Article Outline

Applicability of findings

Findings from the included evidence were considered for applicability to IIS as implemented in the United States. Most included studies (n = 209) described or evaluated US systems, with 166 reporting vaccination outcomes. Published studies included 1 or more assessments from 32 of 53 US systems (60%). Sentinel Site programs (IISs with higher levels of data completeness, provider participation, and system capabilities) contributed 108 (52%) of the US studies. Most US studies examined IIS activities regarding children and vaccinations within the childhood series, whereas only 33 of 166 articles (20%) reporting vaccination outcomes described the use of IIS for adolescents and 12 (7%) described the use of IIS for adult populations, with most studies focused on influenza vaccination. Few studies provided any information on demographic characteristics of the study population such as race/ethnicity, income, education, or insurance status.

Overall, the included studies provide evidence on capabilities that should be applicable to most US IISs, especially as systems achieve levels of provider and client participation demonstrated by Sentinel Site programs. Evidence is broadly applicable to IIS activities regarding vaccinations as part of the childhood series.

Many studies highlighted the benefits of an IIS for tracking vaccine delivery to children eligible for the VFC program; VFC offers no-cost vaccines to children and adolescents through 18 years of age who are uninsured, underinsured, or Medicaid-enrolled or are American Indian or Alaskan Native. Providers participating in the VFC program are required to document the eligibility of participants and track all administered VFC doses, a requirement that is often fulfilled by using an IIS. IISs are also being used to track vaccine inventory, a benefit that can be realized among VFC and non-VFC participating providers, as described in several included studies. As more providers make improvements in vaccine accountability and inventory management, the applicability of IIS to fully represent all children, regardless of eligibility status, is also improved.

Back to Top | Article Outline

Other benefits and harms

In addition to the benefits realized through integration of IIS with other child health information systems (eg, newborn screening, body mass index assessments), described earlier, there are benefits that may come with the ongoing expansion of IIS to life span systems. As more IISs incorporate vaccine administration data for adults, either through enrolling vaccination providers that serve adults or as individuals already enrolled in the IIS age into adulthood, the capabilities of IIS will become more universally applicable to individuals of all ages. Incomplete or delayed reporting of vaccination records might result in overvaccination of some clients with fragmented care, although this potential outcome is not unique to IIS and is likely to be even more common in the absence of a centralized data source.247

Back to Top | Article Outline

Considerations for implementation

As summarized earlier, the studies and outcomes included in this review describe program actions and capabilities that could be adopted or enhanced by other US IISs.

Included studies also describe a number of barriers to the implementation and operation of IISs. Many systems rely on voluntary participation of vaccination providers, with IIS data completeness (and subsequent utility) dependent on the extent and timeliness of provider reporting. In the United States, 31 IIS programs mandate participation by some or all providers by law and only 21 reported a mechanism in place to enforce the mandate.248 In addition, there is variability about which types of providers (ie, public providers, private providers, or VFC providers) and what ages fall under the mandate.248 Vaccination providers have expressed concerns over technological constraints, as well as time and staffing demands required to enter complete and timely reports into the system.249

Client participation and confidentiality requirements may limit the ways in which IIS data can be entered, retrieved, or used for clinical and public health purposes. State policies differ, with 3 IIS programs (Texas, Montana, and South Carolina) requiring written consent to participate in the IIS for all ages (through adulthood) and 8 IIS programs (Arizona, Arkansas, Kansas, Montana, New Jersey, New York City, New York State, and Texas) requiring consent for individuals 19 years and older.248 The requirements of the federal Family Educational and Rights to Privacy Act (FERPA) (http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html) also restrict opportunities to update IIS records using information provided to schools because of requirements for parental or adult client consent for such use. Confidentiality requirements can limit provider access to client records across IIS jurisdictions (although different city and state programs do have data exchange agreements or provide clients access to hard copies of vaccination records), thereby limiting the potential benefits of IIS.

Integration of IIS with other child health information systems (eg, newborn screening) can provide additional benefits by consolidating health information within a single system. Integrated child health information systems reduce duplication of data collected and improve the coordination of services provided, ensuring that all children receive needed preventive care services and case management.250

Back to Top | Article Outline

Conclusion

The studies included in this review provide evidence that IISs are effective in improving vaccination-related activities linked to increased vaccination rates and reduced risk for vaccine-preventable disease. Evidence from 240 published articles and conference abstracts demonstrate IIS capabilities to (1) create or support effective interventions such as client reminder and recall, provider assessment and feedback, and provider reminders (47 studies); (2) determine client vaccination status for decisions made by clinicians, health care systems, and schools (14 studies); (3) generate and evaluate public health responses to outbreaks of vaccine-preventable disease (18 studies); (4) inform assessments of vaccination coverage, missed vaccination opportunities, invalid dose administration, disparities in vaccination rates (110 studies), and vaccine safety and effectiveness (17 studies); and (5) facilitate vaccine management and accountability (14 studies).

Back to Top | Article Outline

Evidence gaps

Only one of the included studies, from Australia, documented the association of improvements in vaccination coverage with the implementation and enhancement of an IIS. Additional studies are needed, especially in US settings, that examine the overall impact of IIS (or IIS activities) either on vaccination coverage in the general population or on important target populations with gaps in coverage.

Few studies provided information relevant to an assessment of the utility of IIS for vaccination providers in clinical settings. Descriptions, assessments, and evaluations focused on the utility of, or barriers to, the use of IIS in day-to-day operations of vaccination providers in a range of clinical settings would provide important information to help determine the overall value of these interventions and identify tools, interactions, and reporting policies in need of adjustment or modification.

As a population-based resource, IIS should capture data and provide services to all members of the population, including groups with lower rates of vaccination coverage. Future studies could examine participation rates and record capture based on demographic characteristics such as race/ethnicity, household income, and insurance status. If IIS records included all data elements specified in the IIS Functional Standards, these analyses could be conducted using solely IIS data. (http://www.cdc.gov/vaccines/programs/iis/func-stds.html). Until IIS consistently capture complete demographic data, this would likely require merging with data from other sources.

Although annual program surveys identify a wide range of IIS-supported actions, such as generation of client reminders, the extent of these activities within the covered population remains unclear. A survey of pediatricians conducted in 2013 by the AAP found that over 70% of respondents reported using their state IIS. Among IIS users, 70% reported using it to recall patients for needed vaccines, and nearly all (97%) said that it reduces unnecessary vaccination.251 Future studies should attempt to quantify the frequency and reach of various IIS activities within the population.

Back to Top | Article Outline

Limitations of this review

The findings of this review are subject to a number of limitations. This project, unlike previous Community Guide reviews, included reports of program descriptions, actions, and experiences and considered this information as evidence contributing to a determination of effectiveness. In most cases, these reports do not provide a formal comparison, although a change in system capability or impact is implied. Of the evidence that included a formal comparison, the most common evaluation was a simple before-after format, a study design that is subject to the influences of potential biases, trends, or other explanatory factors.

Although the search of published evidence was systematic, screening by the team of abstracts from selected US conferences with a known focus on immunization research might have introduced or reinforced a selection bias in the inclusion of evidence. In addition, half of the studies included in this review came from IISs within the US Sentinel Site program, which has requirements to present and submit for publication as a condition of funding. Although Sentinel Sites provide a good understanding of the potential of all IISs, once they have achieved the required levels of participation, these capabilities and findings might not be universally applicable to IIS. However, as more IISs achieve the levels of reporting and timeliness required of Sentinel Sites, the findings from these studies may become more universally applicable.

Back to Top | Article Outline

Discussion

This review provides a systematic and organized synthesis of a large body of evidence on capabilities and effectiveness of IIS. The 240 included studies describe and evaluate a wide range of public health and clinical activities to support public health efforts to conduct assessments of vaccination rates and disparities in coverage, to identify individuals and populations for additional immunization-related intervention, to improve vaccination-related decisions and actions in routine and public health emergency situations, and to support studies assessing vaccine uptake, effectiveness, and safety. Together, the evidence documents utility of IIS for vaccination providers and public health decision makers at the local, state, and national levels and provides a wide range of options for activities to examine and address patient status and local concerns and emergencies.

One primary goal of efforts to implement, populate, and maintain IIS is to support vaccination providers by facilitating (1) vaccination decisions with patients, (2) common immunization-related activities such as generation of vaccination record forms, (3) access to additional interventions to increase vaccination rates and reduce missed opportunities, and (4) interactions with public health and public sector immunization services. Most US state and city IISs include capabilities to generate or support effective interventions to increase vaccination rates, specifically client reminder/recall notices, assessment and feedback activities, and reminder systems for vaccination providers, as suggested by IIS annual report data (see Table 1). However, few studies included in this review provided information about vaccine providers' use of IIS. Some studies have examined effectiveness and use of IIS for generating reminders for pediatric25 and adolescent populations,22 but no formal evaluations examined the frequency of consultation of IIS for clinical decision making within a population of vaccination providers. Without a better understanding of the relationships between providers and these systems in clinical settings, it will remain difficult to (1) assess the utility of IIS for vaccination providers, (2) identify barriers to regular use, and (3) determine how each of the different IIS capabilities contributes to improvements in vaccination rates. The lack of information on the use of IIS in clinical settings is, therefore, especially unfortunate because, if fully used by vaccination providers (or within a participating health care system), these IIS capabilities alone could translate into improved patient follow-up and fewer missed vaccination opportunities.

Studies identified in this review provide some information on the use of IIS by schools and health care organizations. For schools, IIS access provides an efficient mechanism for determining and documenting student vaccination requirements. For health care systems, adoption of IIS as the vaccination database, or incorporation of IIS data into existing EHRs through automated data exchange, can reduce duplication of record keeping, support assessment and feedback initiatives for providers, and facilitate reporting of relevant coverage rates associated with participation in HEDIS. In both settings, participation by an entire system (eg, school district, health care organization) also provides the opportunity to expand enrollment and documentation substantially, further enhancing the completeness and utility of the IIS for the community.

Annual IIS program surveys conducted by the CDC indicate general trends of improved and expanded capabilities of US systems. Table 1 describes the number of immunization programs in the United States that are supported by an IIS that serves their entire state/city jurisdiction and the expanding IIS functionalities that support immunization programs and other immunization stakeholders. These data suggest that programs and partners are increasingly using IIS to facilitate a wide range of interventions, assessments, and response efforts. Table 2 describes population and provider site participation in IIS over time and documents noted improvements in data capture across the life span. In 2002, 64% of IISs were including immunization data for populations of all ages, whereas in 2011, reported inclusion of life span data had increased to 93%. From 2002 to 2011, a 65% increase in the percentage of children participating in IIS and 4-fold increase in provider site participation were observed. It is important to recognize that the definition of child participation in an IIS requires that a child younger than 6 years have a minimum of 2 routinely recommended vaccines entered into an IIS, far fewer vaccines than a child needs to achieve up-to-date status even during infancy. While national trends indicate enhanced IIS functionality and improved data quality, IISs exist at varying levels of maturity, which impacts their ability to support vaccination providers and other immunization stakeholders.

Currently, most IISs in the United States rely on voluntary participation and reporting from vaccination providers to achieve functional levels of data completeness. Without reporting mandates, it is difficult to improve provider participation and timeliness of immunization reporting—issues that directly affect the utility of the system—both for individual clinical care decisions and for population-based assessments and actions. Evidence from Australia's ACIR indicates that IIS can achieve very high levels of participation and data completeness through coordination of system entry and reporting. In Australia, incentives have been offered both to parents whose children were up-to-date with immunizations (or who had filed an exemption) and to vaccination providers based on up-to-date coverage rates within their client population.7 Although coordinated incentives may not be feasible universally at the IIS program level in the United States, elements could be considered for application in some US settings, such as incentives offered by participating health care systems to vaccination providers based on vaccination and reporting performance. In addition, there are substantial barriers to developing a national IIS in the United States, because each immunization program must incorporate unique state or city immunization laws and policies into its IIS functionality, which sometimes create barriers for the exchange of data across state lines.252

While child and provider site participation in IIS is currently high, completeness and timeliness of reporting in US systems may further improve in coming years for a variety of reasons. First, several states have adopted mandatory reporting requirements for some vaccinations, especially for vaccines provided through the VFC program. Second, national, state, and local IIS stakeholders have engaged to develop and publish additional standards to support common IIS functions including CDSi (http://www.cdc.gov/vaccines/programs/iis/interop-proj/cds.html), operational IIS best practice guidelines (http://www.cdc.gov/vaccines/programs/iis/activities/mirow.html), electronic messaging and transport layer standards (http://www.cdc.gov/vaccines/programs/iis/interop-proj/ehr.html), and patient-level deduplication guidelines. The publication and implementation of these standards allow IIS to consolidate knowledge from diverse systems, support consistent best practices across the IIS community, more effectively and efficiently exchange data with other electronic data systems, and reduce costs required for each IIS to address tasks individually. Third, several initiatives have provided federal support for enhanced IIS data use and system functionality. Immunization programs that receive funds under section 317 of the Public Health Service Act are encouraged to use their IIS to assess vaccination coverage for subpopulations (eg, geographic and provider-based populations) to identify pockets of need that might benefit from intervention. To further support enhanced IIS functionality, the CDC has awarded $18.7 million to 37 IISs since 2011 to establish and/or enhance interfaces between IIS and VTrckS, CDC's national vaccine ordering and inventory management system for publicly purchased vaccine. Four additional programs have committed to using their IISs to interface with VTrckS. In these states and cities, vaccination providers who wish to participate in the VFC program order and manage publicly purchased vaccine through the IIS.253 Finally, the modernization of health care in the United States through the electronic exchange of data to support clinical care is anticipated to positively impact IIS. The use of 2-dimensional barcode technology to record and electronically report vaccine information, including product and manufacturer, lot expiration date, and lot number, to EHRs and IIS is expected to further improve IIS data quality.6

Several initiatives have been established to support interoperability in the United States. The ARRA investment in the Section 317 Immunization Program was provided to expand access to vaccines and vaccination services by making more vaccines available, increase national public awareness and knowledge about the benefits and risks of vaccines and vaccine-preventable diseases, and strengthen the evidence base for vaccination policies and programs. The Health Information Technology for Clinical and Economic Health (HITECH) Act includes provisions intended to “accelerate adoption of nationally certified EHR systems, standardize EHR products, support growth in the health information technology workforce, and facilitate secure exchange of health data between disparate partners.”254(p11) The Prevention and Public Health Fund (PPHF),255 created by the Affordable Care Act, was established to invest in prevention and public health programs to improve health and help limit the growth in health care costs. And, the Centers for Medicare & Medicaid Services established an initiative that provides financial incentives to eligible health care providers that acquire certified EHR products and demonstrate meaningful use of these products, defined in part as the exchange of data with IIS (https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Meaningful_Use.html). From 2010 to July 2013, the CDC awarded $46.2 million of American Recovery Reinvestment Act/HITECH and PPHF funds to 37 IISs to enhance interoperability between EHRs and IIS and support the meaningful use initiative. It is anticipated that these activities will result in more complete immunization histories in IIS and improve the timeliness of data submissions to IIS.

With all US jurisdictions, excluding 1 state, supported by an IIS, the findings of this review provide information on the use of programs on efforts to support, expand, enhance, evaluate, or revise their own operations and to improve interactions with vaccination providers and patients.

Back to Top | Article Outline

REFERENCES

1. Centers for Disease Control and Prevention. Ten great public health achievements—United States, 2001-2010. MMWR Morb Mortal Wkly Rep. 2011;60(19):619–623.
2. Centers for Disease Control and Prevention. Ten great public health achievements—United States, 1900-1999. MMWR Morb Mortal Wkly Rep. 1999;48(12):241–243.
3. Hinman AR, Urquhart GA, Strikas RA. Immunization information systems: National Vaccine Advisory Committee progress report, 2007. J Public Health Manag Pract. 2007;13(6):553–558.
4. Yusuf H, Adams M, Rodewald L, et al. Fragmentation of immunization history among providers and parents of children in selected underserved areas. Am J Prev Med. 2002;23(2):106–112.
5. Centers for Disease Control and Prevention. Progress in immunization information systems—United States, 2011. MMWR Morb Mortal Wkly Rep. 2013;62(03):48–51.
6. National Vaccine Advisory Committee. Development of Community- and State-Based Immunization Registries. Washington, DC: National Vaccine Advisory Committee; 1999.
7. Hull B, Dey A, Mahajan D, Menzies R, McIntyre R. Immunisation coverage annual report, 2009. Commun Dis Intell. 2011;35(2):14132–14148.
8. Briss PA, Zaza S, Pappaioanou M, et al. Developing an evidence-based Guide to Community Preventive Services—methods. The Task Force on Community Preventive Services. Am J Prev Med. 2000;18(1)(suppl):35–43.
9. Hull BP, Deeks SL, McIntyre PB. The Australian Childhood Immunisation Register—a model for universal immunisation registers? Vaccine. 2009;27(37):5054–5060.
10. Mennito SH, Darden PM. Impact of practice policies on pediatric immunization rates. J Pediatr. 2010;156:618–622.
11. Allison MA, Daley MF, Barrow J, et al. High influenza vaccination coverage in children with high-risk conditions during a vaccine shortage. Arch Pediatr Adolesc Med. 2009;163(5):426–431.
12. Daley MF, Steiner JF, Brayden RM, Xu S, Morrison S, Kempe A. Immunization registry-based recall for a new vaccine. Ambul Pediatr. 2002;2(6):438–443.
13. Daley MF, Barrow J, Pearson K, et al. Identification and recall of children with chronic medical conditions for influenza vaccination. Pediatrics. 2004;113(1, pt 1):e26–e33.
14. Dombkowski KJ, Harrington LB, Dong S, Clark SJ. Seasonal influenza vaccination reminders for children with high-risk conditions: a registry-based randomized trial. Am J Prev Med. 2012;42(1):71–75.
15. Hambidge SJ, Davidson AJ, Phibbs SL, et al. Strategies to improve immunization rates and well-child care in a disadvantaged population: a cluster randomized controlled trial. Arch Pediatr Adolesc Med. 2004;158(2):162–169.
16. Hambidge SJ, Phibbs SL, Chandramouli V, Fairclough D, Steiner JF. A stepped intervention increases well-child care and immunization rates in a disadvantaged population. Pediatrics. 2009;124(2):455–464.
17. Irigoyen MM, Findley S, Wang D, et al. Challenges and successes of immunization registry reminders at inner-city practices. Ambul Pediatr. 2006;6(2):100–104.
18. Kempe A, Daley MF, Barrow J, et al. Implementation of universal influenza immunization recommendations for healthy young children: results of a randomized, controlled trial with registry-based recall. Pediatrics. 2005;115(1):146–154.
19. LeBaron CW, Starnes DM, Rask KJ. The impact of reminder-recall interventions on low vaccination coverage in an inner-city population. Arch Pediatr Adolesc Med. 2004;158(3):255–261.
20. Melinkovich P, Hammer A, Staudenmaier A, Berg M. Improving pediatric immunization rates in a safety-net delivery system. Jt Comm J Qual Patient Saf. 2007;33(4):205–210.
21. Papadouka V, Metroka A, Zucker JR. Using an immunization information system to facilitate a vaccine recall in New York City, 2007. J Public Health Manag Pract. 2011;17(6):565–568.
22. Saville AW, Albright K, Nowels C, et al. Getting under the hood: exploring issues that affect provider-based recall using an immunization information system. Acad Pediatr. 2011;11(1):44–49.
23. Suh CA, Saville A, Daley MF, et al. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012;129(6):e1437–e1445.
24. Dombkowski KJ. Centralized immunization recall in a large urban area. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
25. Dombkowski KJ. Effectiveness of population-based immunization recall in a large urban area. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
26. Fowler L. New avenues for promoting pediatric immunizations: advice for implementing and Autodialer Awareness Campaign. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
27. Stockwell MS. Text4Health: effectiveness of immunization registry-linked text message reminders on adolescent immunization coverage. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
28. Stockwell MS. FluText: using text messaging to promote timely flu vaccination in an undeserved child population. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
29. Beaudrault S. AFIX and managed care in Oregon: applying the AFIX model in a non-traditional setting. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
30. Cherry L. Using immunization registry reminder-recall reports to improve CASA audit results. Paper presented at: 39th National Immunization Conference; March 20-24, 2005; Washington, DC.
31. Cherry Kreger L. Reminder postcards vs. reminder recall/broadcast telephone 880 technology as a cost saving alternative. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
32. Cushon J. Improving coverage rates and tackling disparities: a Canadian immunization intervention. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
33. Dombkowski KJ. Pandemic preparedness: identifying and reaching priority cases using an IIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
34. Dombkowski KJ. Strategies to leverage IIS capabilities to improve outreach to children with chronic conditions. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
35. Kurilo MB. Using Oregon's IIS to recall children who received subpotent DTaP vaccine. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
36. Muscoplat M. Patient follow-up in high risk areas using IIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
37. Niedner D. Using a state immunization registry to identify parents with invalid vaccinations. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
38. Rasmussen L. Use of the Arizona State IIS (ASIIS) during the Pima County (Tucson, Arizona) measles outbreak. In: SnapShots. New York, NY: American Immunization Registry Association; 2008:8.
39. Sabnis SS. Improving influenza vaccination rates: the effectiveness of intervention at an inner-city academic practice. Paper presented at: Pediatric Academic Societies annual meeting; April 29-May 2, 2006; San Francisco, CA.
40. Zimmerman CM. The epidemiology of measles in New York City (NYC) in 2008 and the use of an immunization information system (IIS) in outbreak control. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
41. Ali H, Zwar N, Wild J. Improving childhood immunisation coverage rates—evaluation of a divisional program. Aust Fam Phys. 2009;38(10):833–835.
42. Jones KL, Hammer AL, Swenson C, et al. Improving adult immunization rates in primary care clinics. Nurs Econ. 2008;26(6):404–407.
43. Baker J. Idaho's Immunization Reminder Information Systems (IRIS) annual data quality audit. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
44. Fernandez K. Implementation of immunization assessments using an IIS in NYC. Paper presented at: Vaccine University; May 28-30, 2008; Baltimore, MD.
45. Francis-Crick P. Restructuring AFIX and VFC program to improve the quality of site visits. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
46. Harris RS. Streamlining AFIX and VFC site visit processes using an IIS. Paper presented at: Vaccine University; May 28-30, 2008; Baltimore, MD.
47. Irigoyen M. Registry-based simulation of chart audits for immunization practice assessments. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2003; Seattle, WA.
48. Johnson AL. Auditing techniques for ensuring quality data in a registry. Paper presented at: Immunization Registry Conference of CDC; October 28-30, 2002; Philadelphia, PA.
49. Mickle-Hope MA. Using an IIS to assess immunization coverage: the NYC experience. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
50. Potter R. An IIS and provider education partnership: results of a pilot study. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
51. Rosen D. A quality improvement system to reduce the rate of invalid immunizations. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
52. Sanchez SJ. Integrating IIS with AFIX to improve feedback and follow-up. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
53. Scholz S. Increasing Childhood Immunization Rates. Milwaukee, WI: Aurora Health Care; 2000.
54. Wolicki J. Using IIS to educate immunization providers on safe and effective vaccine administration. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
55. Chamberlain A. Improving communication during a vaccine shortage: lessons learned from the 2008-2009 Haemophilus influenzae type B (Hib) shortage. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
56. Chamberlain A. Use of IIS by VFC provider sites in Oregon and Louisiana in planning for and managing the 2009-2010 influenza vaccination campaign. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
57. Fu LY. Improving immunization rates of underserved children. Paper presented at: Pediatric Academic Societies annual meeting; May 2-5, 2009; Baltimore, MD.
58. Balog JL. School nurses and immunization registries: a partnership for success. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
59. Cierzniewski A. Impact of state immunization registry on immunization rates of sixth grade students. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
60. Flenner M. Utilizing registry functionality and data for statewide school and childcare reporting—first year results. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
61. Joyce CJ. Using registries to improve employee health immunization management. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
62. Korten LZ. Sharing immunization data with health systems. Paper presented at: Immunization Registry Conference of CDC; October 28-30, 2002; Philadelphia, PA.
63. Kurilo MB. Oregon's ALERT immunization information system: striving to meet the needs of school law. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
64. McCall WW. The easy button: collaborating with statewide immunization information systems to reduce the stress of the “Back to School” Rush. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
65. O'Connor AC, Layton CM, Osbeck TJ, Hoyle TM, Rasulnia B. Health plan use of immunization information systems for quality measurement. Am J Manag Care. 2010;16(3):217–224.
66. Pierce A. Matching school immunization records to NDIIS data. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
67. Sans D. Using incentive programs to improve private provider participation in registry programs. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
68. Sayer N. Keeping track of Idaho's TOTS. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
69. Wake L. Comparing school immunization records to provider records. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
70. Wake L. Comparing school immunization exemptions to registry records. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
71. Wester R. Streamlining school immunization compliance reporting through an IIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
72. Bronson-Lowe D. Effects of recommending a minimum interval schedule for DTaP during a statewide pertussis outbreak in Arizona. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
73. Gaudino JA. Do providers listen to local public health? Evaluating the efficacy of accelerated DTaP immunization recommendations during 2003 pertussis outbreaks in Oregon. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
74. Gosney K. Use of a registry to measure the uptake of an accelerated schedule for DTaP immunization in response to state recommendation during the 2005 pertussis outbreak in Arizona. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
75. Centers for Disease Control and Prevention. Pertussis outbreak in an Amish community—Kent County, Delaware, September 2004-February 2005. MMWR Morb Mortal Wkly Rep. 2006;55(30):817–821.
76. Boom JA, Dragsbaek AC, Nelson CS. The success of an immunization information system in the wake of Hurricane Katrina. Pediatrics. 2007;119(6):1213–1217.
77. Cherry Kreger L. Just in time marketing and training for H1N1 Allows providers to test drive registry software. Paper presented at: National Immunization Conference; April 19-20, 2010; Atlanta, GA.
78. Crielly AS. Improving mass immunizations using a Web-based registry. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
79. Denious A. Impact of registry data during a mass immune globulin vaccination clinic. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
80. Erhart LM. Uptake of Tdap during a statewide outbreak of pertussis, Arizona 2005. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
81. Metroka A. Accounting for H1N1 vaccine using the NYC IIS. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
82. Santilli L. Bringin' them out of the woodworks: H1N1 and NYSIIS participation. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
83. Scott JW. The River Center Shelter Mass Vaccination Clinic. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
84. Tropper J. Monitoring influenza A (H1N1) monovalent vaccine doses administered using CDC's countermeasure and response administration system: lessons learned. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
85. Urquhart GA, Williams W, Tobias J, Welch FJ. Immunization information systems use during a public health emergency in the United States. J Public Health Manag Pract. 2007;13(5):481–485.
86. Zucker JR. H1N1 influenza vaccine distribution and reporting in NYC. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
87. Bardi J. Vaccine ordering and accountability system: integration with the CHILD Profile Immunization Registry (CPIR). Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
88. Fath J. Interfacing external information systems with VTrckS—lesson learned from the 2010 pilot test. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
89. Halsell C. Implementation of online vaccine ordering in Washington State. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
90. Hansen MA. Preparing for VTrckS: implementing an online ordering and management tool in an IIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
91. Hicks-Thompson J. Washington VTrckS ExIS integration: partnership and implementation. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
92. LeBeau T. VFC vaccine accountability through a statewide registry: South Dakota's experience. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
93. Maerz TR. Streamlining vaccine ordering and improving VFC accountability—the role of registries. Paper presented at: 39th National Immunization Conference; March 20-24, 2005; Washington, DC.
94. Metroka AE, Hansen MA, Papadouka V, Zucker JR. Using an immunization information system to improve accountability for vaccines distributed through the Vaccines for Children program in New York City, 2005-2008. J Public Health Manag Pract. 2009;15(5):E13–E21.
95. Philbrick T. Accountability thorough technology: making every dose count. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
96. Reader-Jolley D. Statewide rollout of online vaccine ordering: lessons learned and best practices. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
97. Sander M. Lessons learned: billing insurance at local public health units in North Dakota. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
98. Swanson R. Michigan's experience implementing VTrckS using EXIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
99. Tichy K. An integrated approach to a statewide IIS. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
100. Allison MA, Daley MF, Crane LA, et al. Influenza vaccine effectiveness in healthy 6- to 21-month-old children during the 2003-2004 season. J Pediatr. 2006;149(6):755–762.
101. Boom JA, Tate JE, Sahni LC, et al. Effectiveness of pentavalent rotavirus vaccine in a large urban population in the United States. Pediatrics. 2010;125(2):e199–e207.
102. Cortese MM, Leblanc J, White KE, et al. Leveraging state immunization information systems to measure the effectiveness of rotavirus vaccine. Pediatrics. 2011;128(6):e1474–e1481.
103. Gold M, Dugdale S, Woodman RJ, McCaul KA. Use of the Australian Childhood Immunisation Register for vaccine safety data linkage. Vaccine. 2010;28(26):4308–4311.
104. Guh AY, Hadler JL. Use of the state immunization information system to assess rotavirus vaccine effectiveness in Connecticut, 2006-2008. Vaccine. 2011;29(37):6155–6158.
105. Hadler J. Effectiveness of one dose of 2009 monovalent H1N1 vaccine at preventing hospitalization with pandemic H1N1 influenza in children 7 months to 9 years of age, New York City. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
106. Mahon BE, Shea KM, Dougherty NN, Loughlin AM. Implications for registry-based vaccine effectiveness studies from an evaluation of an immunization registry: a cross-sectional study. BMC Public Health. 2008;8:160.
107. Markey P, Krause V, Boslego JW, Coplan PM, Dargan JM, Kaplan KM. The effectiveness of Haemophilus influenzae type B conjugate vaccines in a high risk population measured using immunization register data. Epidemiol Infect. 2001;126(1):31–36.
108. Muhammad RD. Using IIS to evaluate effectiveness of pneumococcal conjugate vaccine among children. Paper presented at: 59th EIS Conference; April 19-23, 2010; Atlanta, GA.
109. O'Hallahan J, McNicholas A, Galloway Y, O'Leary E, Roseveare C. Delivering a safe and effective strain-specific vaccine to control an epidemic of group B meningococcal disease. N Z Med J. 2009;122(1291):48–59.
110. Placzek H, Madoff LC. The use of immunization registry-based data in vaccine effectiveness studies. Vaccine. 2011;29(3):399–411.
111. Ryan MA, Smith TC, Sevick CJ, et al. Birth defects among infants born to women who received anthrax vaccine in pregnancy. Am J Epidemiol. 2008;168(4):434–442.
112. Sahni LC, Boom JA, Patel MM, et al. Use of an immunization information system to assess the effectiveness of pentavalent rotavirus vaccine in US children. Vaccine. 2010;28(38):6314–6317.
113. Salmon DA, Akhtar A, Mergler MJ, et al. Immunization-safety monitoring systems for the 2009 H1N1 monovalent influenza vaccination program. Pediatrics. 2011;127(suppl 1):S78–S86.
114. Schauer SL. Evaluation of the Impact of Rotavirus Vaccine on Acute Care Visits, Hospitalizations and Laboratory Testing in Wisconsin, 2002-2008. Washington, DC: APHA; 2009.
115. Tatley MV, Kunac DL, McNicholas A, et al. The Intensive Vaccines Monitoring Programme (IVMP): an electronic system to monitor vaccine safety in New Zealand. Vaccine. 2008;26(22):2746–2752.
116. Zucker JR. Use of an IIS to facilitate timely recall of potentially contaminated vaccine. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
117. Phibbs SL, Hambidge SJ, Steiner JF, Davidson AJ. The impact of inactive infants on clinic-based immunization rates. Ambul Pediatr. 2006;6(3):173–177.
118. Adamkiewicz TV, Silk BJ, Howgate J, et al. Effectiveness of the 7-valent pneumococcal conjugate vaccine in children with sickle cell disease in the first decade of life. Pediatrics. 2008;121(3):562–569.
119. Allison MA, Crane LA, Beaty BL, Davidson AJ, Melinkovich P, Kempe A. School-based health centers: improving access and quality of care for low-income adolescents. Pediatrics. 2007;120(4):e887–e894.
120. Allred NJ, Stevenson JM, Kolasa M, et al. Using registry data to evaluate the 2004 pneumococcal conjugate vaccine shortage. Am J Prev Med. 2006;30(4):347–350.
121. Bond L, Davie G, Carlin JB, Lester R, Nolan T. Increases in vaccination coverage for children in child care, 1997 to 2000: an evaluation of the impact of government incentives and initiatives. Aust N Z J Public Health. 2002;26(1):58–64.
122. Botham SJ, Poulos RG, McFarland KJ, Ferson MJ. Getting it right—the Australian Childhood Immunisation Register and immunisation rates in south-eastern Sydney. Aust N Z J Public Health. 2004;28(1):68–71.
123. Boulton ML, Grossman AM, Potter R, Vranesich PA, Clayton J. Assessing the relationship between seasonal and H1N1 influenza vaccination status in Michigan children, 2009-2010. Public Health Rep. 2011;126(suppl 2):70–77.
124. Bronson-Lowe D, Anderson SM. Effects of a minimum interval immunization schedule for diphtheria and tetanus toxoids and acellular pertussis vaccination during a pertussis outbreak. Arch Pediatr Adolesc Med. 2009;163(5):417–421.
125. Callahan JM, Reed D, Meguid V, Wojcik S, Reed K. Utility of an immunization registry in a pediatric emergency department. Pediatr Emerg Care. 2004;20(5):297–301.
126. Cugley K, Crawford N, Royle J, Elia S, Massie J. Immunisation rates of children with cystic fibrosis using the Australian Childhood Immunisation Register. J Paediatr Child Health. 2010;46(12):768–771.
127. Daskalaki I, Spain CV, Long SS, Watson B. Implementation of rotavirus immunization in Philadelphia, Pennsylvania: high levels of vaccine ineligibility and off-label use. Pediatrics. 2008;122(1):e33–e38.
128. Enger KS, Stokley S. Meningococcal conjugate vaccine uptake, measured by Michigan's immunization registry. J Adolesc Health. 2007;40(5):398–404.
129. Ernst KC, Pogreba-Brown K, Rasmussen L, Erhart LM. The effect of policy changes on hepatitis a vaccine uptake in Arizona children, 1995-2008. Public Health Rep. 2011;126(suppl 2):87–96.
130. Feemster KA, Spain CV, Eberhart M, Pati S, Watson B. Identifying infants at increased risk for late initiation of immunizations: maternal and provider characteristics. Public Health Rep. 2009;124(1):42–53.
131. Findley SE, Irigoyen M, Sanchez M, et al. Effectiveness of a community coalition for improving child vaccination rates in New York City. Am J Public Health. 2008;98(11):1959–1962.
132. Giraudon I, Permalloo N, Nixon G, et al. Factors associated with incomplete vaccination of babies at risk of perinatal hepatitis B transmission: a London study in 2006. Vaccine. 2009;27(14):2016–2022.
133. Goldstein ND. A brief review of vaccination coverage in immunization registries. Online J Public Health Inform. 2011;3(1):1–7.
134. Hanna JN, Bullen RC, Ziegler CL, Akee T, Dostie BG, Lort-Phillips K. An assessment of the implementation of the pneumococcal conjugate vaccination program for Aboriginal and Torres Strait infants in north Queensland. Commun Dis Intell Q Rep. 2003;27(2):262–266.
135. Hanna JN, Bullen RC, Andrews DE. The acceptance of three simultaneous vaccine injections recommended at 12 months of age. Commun Dis Intell Q Rep. 2004;28(4):493–496.
136. Hanna JN, Hills SL, Humphreys JL. Impact of hepatitis a vaccination of Indigenous children on notifications of hepatitis a in north Queensland. Med J Aust. 2004;181(9):482–485.
137. Hanna JN, Morgan AK, McCulloch BG. Uptake of influenza vaccine among Aboriginal and Torres Strait Island adults in north Queensland, 2003. Commun Dis Intell. 2004;28(1):80–82.
138. Happe LE, Lunacsek OE, Marshall GS, Lewis T, Spencer S. Combination vaccine use and vaccination quality in a managed care population. Am J Manag Care. 2007;13(9):506–512.
139. Hull BP, McIntyre PB. A re-evaluation of immunisation coverage estimates from the Australian Childhood Immunisation Register. Commun Dis Intell. 2000;24(6):161–164.
140. Hull BP, McIntyre PB. Immunisation coverage reporting through the Australian Childhood Immunisation Register—an evaluation of the third-dose assumption. Aust N Z J Public Health. 2000;24(1):17–21.
141. Hull BP, McIntyre PB, Sayer GP. Factors associated with low uptake of measles and pertussis vaccines—an ecologic study based on the Australian Childhood Immunisation Register. Aust N Z J Public Health. 2001;25(5):405–410.
142. Hull B, McIntyre P. Mapping immunisation coverage and conscientious objectors to immunisation in NSW. N S W Public Health Bull. 2003;14(1–2):8–12.
143. Hull BP, Lawrence GL, MacIntyre CR, McIntyre PB. Immunisation coverage in Australia corrected for under-reporting to the Australian Childhood Immunisation Register. Aust N Z J Public Health. 2003;27(5):533–538.
144. Hull BP, McIntyre PB. What do we know about 7vPCV coverage in Aboriginal and Torres Strait Islander children? Commun Dis Intell. 2004;28(2):238–243.
145. Hull BP, McIntyre PB. Timeliness of childhood immunisation in Australia. Vaccine. 2006;24(20):4403–4408.
146. Lawrence GL, MacIntyre CR, Hull BP, McIntyre PB. Measles vaccination coverage among five-year-old children: implications for disease elimination in Australia. Aust N Z J Public Health. 2003;27(4):413–418.
147. Lawrence GL, Hull BP, MacIntyre CR, McIntyre PB. Reasons for incomplete immunisation among Australian children. A national survey of parents. Aust Fam Phys. 2004;33(7):568–571.
148. Lawrence GL, MacIntyre CR, Hull BP, McIntyre PB. Effectiveness of the linkage of child care and maternity payments to childhood immunisation. Vaccine. 2004;22(17/18):2345–2350.
149. Lister S, McIntyre PB, Burgess MA, O'Brien ED. Immunisation coverage in Australian children: a systematic review 1990-1998. Commun Dis Intell. 1999;23(6):145–170.
150. LoMurray K, Sander M. Using the North Dakota Immunization Information System to determine adolescent vaccination rates and uptake. Public Health Rep. 2011;126(suppl 2):78–86.
151. McIntyre PB, Heath TC, O'Brien ED, Hull BP. National immunisation coverage—interpreting the first three quarterly reports from the ACIR. Commun Dis Intell. 1998;22(6):111–112.
152. Ma KK, Schaffner W, Colmenares C, Howser J, Jones J, Poehling KA. Influenza vaccinations of young children increased with media coverage in 2003. Pediatrics. 2006;117(2):e157–e163.
153. Malcolm RL, Ludwick L, Brookes DL, Hanna JN. The investigation of a ‘cluster’ of hepatitis B in teenagers from an indigenous community in North Queensland. Aust N Z J Public Health. 2000;24(4):353–355.
154. Centers for Disease Control and Prevention. Influenza vaccination coverage among children aged 6-59 months—six immunization information system Sentinel Sites, United States, 2006-07 influenza season. MMWR Morb Mortal Wkly Rep. 2007;56(37):963–965.
155. Centers for Disease Control and Prevention. Influenza vaccination coverage among children aged 6-59 months—eight immunization information system Sentinel Sites, United States, 2007-08 influenza season. MMWR Morb Mortal Wkly Rep. 2008;57(38):1043–1046.
156. Centers for Disease Control and Prevention. Influenza vaccination coverage among children aged 6 months-18 years—eight immunization information system Sentinel Sites, United States, 2008-09 influenza season. MMWR Morb Mortal Wkly Rep. 2009;58(38):1059–1062.
157. Centers for Disease Control and Prevention. Seasonal influenza vaccination coverage among children aged 6 months-18 years—eight immunization information system Sentinel Sites, United States, 2009-10 influenza season. MMWR Morb Mortal Wkly Rep. 2010;59(39):1266–1269.
158. Cullen K. Simultaneous administration among females of HPV4 vaccine and other vaccines recommended for adolescents—IIS Sentinel Sites, United States, 2008-2009. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
159. Centers for Disease Control and Prevention. Rotavirus vaccination coverage among infants aged 5 months—immunization information system Sentinel Sites, United States, June 2006-June 2009. MMWR Morb Mortal Wkly Rep. 2010;59(17):521–524.
160. Olson JJ, Mannenbach MS, Moore BR, Smith VD, Rosekrans JA, Jacobson RM. A reexamination of the feasibility of the administration of routine childhood vaccines in emergency departments in the era of electronic vaccine registries. Pediatr Emerg Care. 2005;21(9):565–567.
161. Ortega AN, Stewart DC, Dowshen SA, Katz SH. The impact of a pediatric medical home on immunization coverage. Clin Pediatr (Phila). 2000;39(2):89–96.
162. Robison SG, Kurosky SK, Young CM, Gallia CA, Arbor SA. Immunization milestones: a more comprehensive picture of age-appropriate vaccination. J Biomed Biotechnol. 2010;2010:916525.
163. Schauer SL, Maerz TR, Hurie MB, Gabor GW, Flynn JM, Davis JP. The use of an immunization information system to establish baseline childhood immunization rates and measure contract objectives. J Public Health Manag Pract. 2009;15(5):E6–E12.
164. Stockwell MS, Brown J, Chen S, Vaughan RD, Irigoyen M. Is underimmunization associated with child maltreatment? Ambul Pediatr. 2008;8(3):210–213.
165. Torvaldsen S, Hull BP, McIntyre PB. Using the Australian Childhood Immunisation Register to track the transition from whole-cell to acellular pertussis vaccines. Commun Dis Intell. 2002;26(4):581–583.
166. Vlack S, Foster R, Menzies R, Williams G, Shannon C, Riley I. Immunisation coverage of Queensland indigenous two-year-old children by cluster sampling and by register. Aust N Z J Public Health. 2007;31(1):67–72.
167. White KE, Anderson J, Stanley M, Ehresmann K. Evaluating hepatitis B universal birth dose vaccination at Minnesota birthing hospitals by utilizing immunization information systems, birth certificates, and chart reviews, 2007-2008. J Public Health Manag Pract. 2009;15(6):464–470.
168. White KE, Pabst LJ, Cullen KA. Up-to-date Haemophilus influenzae type b vaccination coverage during a vaccine shortage. Pediatrics. 2011;127(3):e707–12.
169. Yawn BP, Schroeder C, Wollan P, Rocca L, Zimmerman R, Bardenheier B. Immediate and longer term impact of the varicella shortage on children 18 and 24 months of age in a community population. BMC Fam Pract. 2006;7:51.
170. Bailowitz A. Pediatric influenza immunization in Baltimore City, 1999-2004. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
171. Bartlett DL. Uptake of human papillomavirus vaccine by age group—IIS Sentinel Sites, August 2006-2007. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
172. Bartlett DL. Assessing the number of vaccines received per visit and frequency of vaccination visits among fully vaccinated and partially vaccinated children 16-19 months of age, IIS Sentinel Sites 2004-8. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
173. Health Protection Agency. October to December 2008: quarterly vaccination coverage statistics for children aged up to five years in the United Kingdom [COVER programme]. Health Protection Rep. March 27, 2009;6(12).
174. Cullen K. Newborn hepatitis B vaccination coverage among children aged 3, 5, and 19-35 months—IIS Sentinel Sites, United States 2008-9. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
175. Cullen K. Factors associated with reporting a child's vaccination history to an IIS, National Immunization Survey 2006-2008. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
176. Daskalaki I. Implementation of the 2-dose varicella vaccination regimen among children aged 4-6 years in Philadelphia: a good start but a long way to go. Paper presented at: Pediatric Academic Society annual meeting; May 2-5, 2009; Baltimore, MD.
177. Eberhart M. Impact of outreach activities on childhood immunization coverage in Philadelphia. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
178. Enger KS. Replacement of Td with Tdap in Michigan in 2006, measured using the MCIR. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
179. Enger KS. Immunization exemptions in Michigan, 2005-2006 and 2006-2007 school years. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
180. Federico S. Overcoming immunization disparities: a retrospective analysis of school-based health center immunization delivery. Paper presented at: Pediatric Academic Society annual meeting; May 2-5, 2009; Baltimore, MD.
181. Findley S. Use of an immunization registry to enhance outreach and tracking by community organizations. Immunization Registry Conference of CDC; October 28-30, 2002; Philadelphia, PA.
182. Gaudino JA. Monitoring new adolescent immunization uptake using IIS: early trends from Oregon's CDC Sentinel IIS surveillance population. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
183. Gosney K. Comparison of immunization completion rates between American Indian/Alaska Native (AI/AN) and non-AI/AN children in the Arizona Sentinel Site study region. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
184. Groom H. Adolescent vaccination uptake among students participating in Tdap-only clinics in Deschutes County, Oregon. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
185. Gubernick RS. Continuing to achieve, sustain, and spread improvements in immunization delivery. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
186. Hamstra S. Comparing vaccine providers: effect on 43133 immunization rates for AI/AN children in Arizona. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
187. Hanna M. Evaluation of the delivery of immunizations to children under supervision of the Department of Children and Family Services in Southern Los Angeles County. Paper presented at: Pediatric Academic Societies annual meeting; May 2-5, 2009; Baltimore, MD.
188. Hopfensperger DJ. Using IIS and performance based contracts to improve childhood immunization rates. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
189. Kelly J. Assessing variability among IIS vaccine forecasting algorithms. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
190. Kolasa M. Provider chart audits and outreach to parents: impact in improving childhood immunization coverage and immunization information system completeness. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
191. Kurilo MB. Measuring adolescent data capture in Oregon's IIS Sentinel region. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
192. LeBeau T. Utilizing a statewide immunization information system and GIS mapping software to determine pockets of need. Paper presented at: Vaccine University; May 28-30, 2008; Baltimore, MD.
193. Ledezma E. Characteristics of populations delaying scheduled immunizations within Bexar County, Texas. Paper presented at: 39th National Immunization Conference; March 20-24, 2005; Washington, DC.
194. LoMurray K. Assessing vaccination coverage of American Indian children using the North Dakota immunization information system. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
195. Mosquera MC. Evaluating immunization coverage in NYC by geographic area. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
196. Nguyen M. Mapping immunization rates. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
197. Pabst L. Achieving full vaccination coverage with influenza vaccine among children 6-59 months—IIS Sentinel Sites, 2007-8 influenza season. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
198. Papadouka V. Impact of childhood hepatitis A vaccination. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
199. Pati S. Does maternal health literacy predict immunization status? Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
200. Pitman JP. Linking immunization registries and geographic information systems (GIS) to detect areas of low immunization coverage (PON). Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
201. Powers ME. Immunization rates among children of adolescent mothers. Paper presented at: Pediatric Academic Societies annual meeting; May 5-8, 2007; Toronto, Canada.
202. Rasmussen L. Using Arizona's IIS to collect hepatitis A/B immunizations given to vulnerable populations. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
203. Rasulnia B. Using IIS data to assess clinical and operational processes for improving immunization coverage among 19-35 month olds. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
204. Robison SG. A practical method of estimating county and local immunization rates from the Oregon ALERT registry. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
205. Robison SG. Enhancing state and local immunization surveillance by partnering with payors: an example from Oregon and the Oregon health plan. Paper presented at: 44th National Immunization Conference April 19-20; 2010; Atlanta, GA.
206. Sandler M. An effective early childhood immunization program: one community's fifteen year experience. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta. GA.
207. Sathya B. Vaccination coverage of children in the city of Philadelphia shelter system. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
208. Sawyer M. What's going on out there? Measuring the use of alternative immunization schedules. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
209. Siegler PA. Increasing adult pneumococcal immunization rates through innovative software applications and collaborative efforts. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
210. Skiles MP. Improving immunization rates among WIC clients: how an immunization registry helped. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
211. Slater S. Timely immunization in Texas and barriers parents face. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
212. Stockwell MS. How parents' experiences at immunization visits affects child immunization status. Paper presented at: Pediatric Academic Societies annual meeting; May 2-5, 2009; Baltimore, MD.
213. Thomas C. Evaluating and decreasing disparities using an IIS in a Sentinel Site school-age population. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
214. Wells WH. Utilizing links to improve immunization status of children admitted to the children's hospital of LSUHSC-Shreveport. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
215. Young CM. Immunization rates among Oregon Medicaid and WIC participants. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
216. Zimmerman L. Assessing extraimmunization among children using IIS. Paper presented at: 41st National Immunization Conference; March 5-8, 2007; Kansas City, MO.
217. Cordon A, Costello B, Sheon A. Watching the weight of the community: use an IIS for BMI tracking and intervention. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
218. Fairbrother G, Simpson LA. It is time! Accelerating the use of child health information systems to improve child health. Pediatrics. 2009;123(suppl 2):S61–S63.
219. Hinman AR, Atkinson D, Diehn TN, et al. Principles and core functions of integrated child health information systems. J Public Health Manag Pract. 2004;(suppl):S52–S56.
220. Hoyle T, Swanson R. Assessing what child health information systems should be integrated: the Michigan experience. J Public Health Manag Pract. 2004;(suppl):S66–S71.
221. Hoyle T. What information should be integrated with the childhood immunization registry? Mich Med. 2005;104(1):18–19.
222. Kleyman Z, Aveni K. Expanding IIS to EHR: tracking and reporting early hearing detection and intervention. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
223. Papadouka V, Schaeffer P, Metroka A, et al. Integrating the New York citywide immunization registry and the childhood blood lead registry. J Public Health Manag Pract. 2004;(suppl):S72–S80.
224. Therrell BL Jr. Data integration and warehousing: coordination between newborn screening and related public health programs. Southeast Asian J Trop Med Public Health. 2003;34(suppl 3):63–68.
225. Adams WG, Conners WP, Mann AM, Palfrey S. Immunization entry at the point of service improves quality, saves time, and is well-accepted. Pediatrics. 2000;106(3):489–492.
226. Boyd TD, Linkins RW, Mason K, Bulim I, Lemke B. Assessing immunization registry data completeness in Bexar County, Texas. Am J Prev Med. 2002;22(3):184–187.
227. Czaja C, Crossette L, Metlay JP. Accuracy of adult reported pneumococcal vaccination status of children. Ann Epidemiol. 2005;15(4):253–256.
228. Davidson AJ, Melinkovich P, Beaty BL, et al. Immunization registry accuracy: improvement with progressive clinical application. Am J Prev Med. 2003;24(3):276–280.
229. DeVries OE, Knobloch MJ, Garlitz W, et al. Using an Internet-based registry to link school immunization records with community records. J Sch Health. 2003;73(5):201–202.
230. Kempe A, Steiner JF, Renfrew BL, Lowery E, Haas K, Berman S. How much does a regional immunization registry increase documented immunization rates at primary care sites in rural Colorado? Ambul Pediatr. 2001;1(4):213–216.
231. Kempe A, Beaty BL, Steiner JF, et al. The regional immunization registry as a public health tool for improving clinical practice and guiding immunization delivery policy. Am J Public Health. 2004;94(6):967–972.
232. Khare M, Piccinino L, Barker LE, Linkins RW. Assessment of immunization registry databases as supplemental sources of data to improve ascertainment of vaccination coverage estimates in the national immunization survey. Arch Pediatr Adolesc Med. 2006;160(8):838–842.
233. Kolasa MS, Cherry JE, Chilkatowsky AP, Reyes DP, Lutz JP. Practice-based electronic billing systems and their impact on immunization registries. J Public Health Manag Pract. 2005;11(6):493–499.
234. Renfrew BL, Kempe A, Lowery NE, Chandramouli V, Steiner JF, Berman S. The impact of immunization record aggregation on up-to-date rates—implications for immunization registries in rural areas. J Rural Health. 2001;17(2):122–126.
235. Schwarz K, Garrett B, Lamoreux J, Bowser YD, Weinbaum C, Alter MJ. Hepatitis B vaccination rate of homeless children in Baltimore. J Pediatr Gastroenterol Nutr. 2005;41(2):225–229.
236. Stecher DS, Adelman R, Brinkman T, Bulloch B. Accuracy of a state immunization registry in the pediatric emergency department. Pediatr Emerg Care. 2008;24(2):71–74.
237. Stille CJ, Christison-Lagay J. Determining immunization rates for inner-city infants: statewide registry data vs medical record review. Am J Public Health. 2000;90(10):1613–1615.
238. Andersen AD. Accuracy of documented immunization status in pediatric emergency department patients. Paper presented at: Pediatric Academic Societies annual meeting; May 3-6, 2008; Honolulu, HI.
239. Copeland KA. Comparison of vaccination coverage estimates: NIS vs IIS. Paper presented at: 45th National Immunization Conference; March 28-31, 2011; Washington, DC.
240. Dawson V. Ensuring data quality, New Jersey statewide QA initiative. Paper presented at: 43rd National Immunization Conference; March 30-April 2, 2009; Dallas, TX.
241. Irigoyen M. Point of service versus real time registry-based coverage assessment. Immunization Registry Conference of CDC; October 28-30, 2002; Philadelphia, PA.
242. Papadouka V. Accuracy and completeness of immunization registry data: a comparison with chart data. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
243. Shinall MC Jr, Plosa E, Poehling KA. Validity of parental report of influenza vaccination in children 6 to 59 months of age. Paper presented at: Pediatric Academic Societies annual meeting; April 29-May 2, 2006; San Francisco, CA.
244. Rucinski D. Improving data quality of electronic registries. Paper presented at: 40th National Immunization Conference; March 6-9, 2006; Atlanta, GA.
245. Stockwell MS. Integrating electronic information to improve documentation of immunization rates for a low-income, minority population. Paper presented at: 44th National Immunization Conference; April 19-20, 2010; Atlanta, GA.
246. Zimmerman CM, Mickle-Hope MA, Papadouka V, et al. Validation of an immunization information system against the National Immunization Survey and improvement of hepatitis B birth dose coverage in New York City. Paper presented at: 42nd National Immunization Conference; March 17-20, 2008; Atlanta, GA.
247. Abramson E. Improving Immunization data management: an editorial on the potential of electronic health records. Expert Rev Vacc. 2014;13(2):189–191.
248. Martin DW. Immunization information systems: a decade of progress in law and policy [published online ahead of print January 7, 2014]. J Public Health Manag Pract. doi:10.1097/PHH.0000000000000040.
249. Clark SJ. Private provider participation in statewide immunization registries. BMC Public Health. 2006;6:33.
250. Association of State and Territorial Health Officials. Integrated Child Health Information Systems and Privacy: A Review of Four State Approaches. Arlington, VA: Association of State and Territorial Health Officials; 2005.
251. Almquist J. Survey details pediatricians' knowledge, use of immunization registries. AAP News. 2014;35:33.
252. Hedden EM. Childhood immunization reporting laws in the United States: current status. Vaccine. 2012;30(49):7059–7066.
253. Centers for Disease Control and Prevention. Progress in immunization information systems—United States, 2010. MMWR Morb Mortal Wkly Rep. 2012;61(25):464–467.
254. Centers for Disease Control and Prevention. Progress in immunization information systems—United States, 2009. MMWR Morb Mortal Wkly Rep. 2011;60(1):10–12.
255. Savel TG, Foldy S. The role of public health informatics in enhancing public health surveillance. MMWR Morb Mortal Wkly Rep. 2012;61(3):20–24.
Back to Top | Article Outline

Appendix Search for Evidence

This Task Force finding is based on studies identified in the search period (1994-2011). Reference lists of articles reviewed as well as lists in review articles were also searched, and members of our coordination team were consulted for additional references.

Back to Top | Article Outline

Search Strategy

Details of the search (1994-2011)

The team conducted a broad literature search to identify studies evaluating the effectiveness of an IIS or population-based vaccination database, or the effectiveness of an intervention generated from an IIS, or describe an IIS capability linked to increasing vaccination rates or reducing vaccine-preventable disease. The following 9 databases were searched during the period of January 1994 up to April 2011: CABI, CINAHL, The Cochrane Library, EMBASE, ERIC, MEDLINE, PsycINFO, Sociological Abstracts, and Web of Science. Reference lists of articles reviewed as well as lists in review articles were also searched, and subject matter experts were consulted for additional references. The team also included an assessment of unpublished US literature in the form of conference abstracts. Abstracts presented at 4 selected conferences (Immunization Registry Conference; National Immunization Conference, Pediatric Academic Society, Vaccine University) in the United States were searched during the period January 2002 to April 2011. To be included in the review, a study had to

* have a publication date of 1994-2011;

* evaluate vaccinations recommended for children, adolescents, and/or adults;

* meet the evidence review and Guide chapter development team's definition of the interventions;

* be a primary research publication or US conference abstract with 1 or more outcomes related to the analytic frameworks;

* take place in a high-income country or countries;

* be written in English'

* report 1 or more quantitative outcomes (changes in vaccination rates, coverage, or uptake; reductions in missed opportunities) or describe system capabilities linked to increasing vaccinations or reducing vaccine-preventable disease.

Back to Top | Article Outline

Search Terms

MeSH (Medical Subject Headings) were exploded to include all related subject headings. British spelling variations, plurals, title, abstract, and keywords were searched in the non-MEDLINE databases as well.

1. Immunization

2. Vaccination

3. Immunization information system

4. Immunization registry

5. Vaccination registry

Cited Here...

* Members of the review team were Diana Bartlett, CDC; Ned Calonge, The Colorado Trust; Rebecca Coyle, American Immunization Registry Association (AIRA); Kevin Dombkowski, University of Michigan; Amy Groom, CDC field staff assigned to Indian Health Services; Joe Hagan, Pediatrician; Mary Beth Kurilo, Oregon Immunization Program; Bobby Rasulnia, CDC; Sue Scholz, Aurora Health Care; Timothy Van Wave, CDC; Pascale Wortley, CDC; Jane Zucker, CDC field staff assigned to New York City's Department of Health and Mental Hygiene. Cited Here...

* Countries with high-income economies as defined by the World Bank are Andorra, Antigua & Barbuda, Aruba, Australia, Austria, The Bahamas, Bahrain, Barbados, Belgium, Bermuda, Brunei Darussalam, Canada, Cayman Islands, Channel Islands, Cyprus, Denmark, Estonia, Faeroe Islands, Finland, France, French Polynesia, Germany, Greece, Greenland, Guam, Hong Kong (China), Iceland, Ireland, Isle of Man, Israel, Italy, Japan, Republic of Korea, Kuwait, Liechtenstein, Luxembourg, Macao (China), Malta, Monaco, the Netherlands, Netherlands Antilles, New Caledonia, New Zealand, Norway, Portugal, Puerto Rico, Qatar, San Marino, Saudi Arabia, Singapore, Slovenia, Spain, Sweden, Switzerland, Trinidad and Tobago, United Arab Emirates, United Kingdom, United States, and Virgin Islands (United States). Cited Here...

* References 11, 33, 34, 36, 51, 54, 107, 109, 114, 118–216. Cited Here...

† References 33, 34, 36, 51, 54, 114, 158, 170–216. Cited Here...

‡ References 117, 118, 121, 122, 124, 125, 131–133, 138–140, 143, 145–149, 151–161, 163, 166, 167. Cited Here...

§ References 170, 172–175, 183–185, 187, 188, 191, 197, 199, 201, 203, 205, 212–214. Cited Here...

¶ References 11, 14, 33, 34, 107, 109, 119, 126, 130, 134, 136, 137, 142, 144, 153, 164, 177, 179–181, 183, 186, 190, 193, 198, 202, 206, 207, 209–211, 215. Cited Here...

∥ References 32, 36, 141, 192, 194–196, 200, 204. Cited Here...

# References 109, 123, 128, 129, 134, 135, 137, 144, 150, 165. Cited Here...

* References 106, 190, 225, 226, 228, 229, 233, 235, 237, 238, 240–242, 244, 245. Cited Here...

† References 225–229, 232, 233, 235, 237, 241–244. Cited Here...

immunization registry; immunization information systems; systematic review; vaccination

Copyright © 2015 Wolters Kluwer Health, Inc. All rights reserved.