Estimates of the effectiveness of long-standing vaccination programs provide insight into the value of these programs to public health.1 These insights are especially important for policy makers to motivate the continuation of these interventions in this time of increasing vaccine hesitancy.2 Halloran and Struchiner3 described the overall effectiveness of a vaccination program as the ratio of the observed disease burden in a population with a vaccination program to that in a population without such a program. Such a measure takes both direct and indirect protection into account. Including indirect protection is important as it is the distinguishing feature of most vaccination programs.1,4–7 One approach for estimating vaccine program effectiveness would be to compare the burden in the prevaccination era to the burden shortly after the introduction of the program.8 However, such comparisons for long-standing vaccination programs typically ignore secular trends in disease burden. Another approach is to construct a so-called counterfactual: the expected situation had the vaccination program not been introduced. One can then directly compare the observed disease burden (in the actual situation with an implemented vaccination program) to the expected burden in the same population without a vaccination program.
Constructing a counterfactual is not straightforward as diverse prevaccination dynamics need to be taken into account. This is especially the case for the earlier vaccines, such as diphtheria, pertussis, tetanus, and polio, which were introduced in many developed countries in the mid-20th century. Indeed, few studies have focused on the effectiveness of early mass vaccination programs due to lack of data and proper analysis methods.
Here we examine the population-level overall effectiveness of vaccination programs on mortality burden and show that this overall effectiveness can be partitioned into a direct and indirect components. To do so, we make use of data from the Netherlands, where detailed mortality statistics are available from 1903 onward.9,10 In a previous analysis of these data, we showed that the all-cause childhood mortality burden declined exponentially over the 20th century, and that the mortality burden of many vaccine-preventable diseases declined at a similar exponential rate in the prevaccination period.9 Besides mortality data, information on vaccination coverage is available since the implementation of mass vaccination programs in 1953. This makes the Dutch data uniquely equipped for investigating vaccination program effectiveness.
We pose the following research questions: is there evidence for indirect effects of vaccination programs on mortality burden, and what is the magnitude of these indirect effects? To address these questions, we constructed counterfactual scenarios (i.e., scenarios in which vaccination programs were not implemented) using a model to describe trends in the prevaccination era, and estimated program effectiveness with respect to childhood mortality burden in the first 2 decades after the start of mass vaccination in the 1950s. We quantified the magnitude of direct and indirect effects for three communicable vaccine-preventable diseases (diphtheria, pertussis, and polio), and one noncommunicable vaccine-preventable disease (tetanus), which serves as a negative control.11
METHODS
Childhood Mortality Burden
We used data on vaccination coverage and cause-specific mortality as previously reported in van Wijhe et al.9 Briefly, these data, spanning the period 1903–2012, were obtained from the national census bureau (Statistics Netherlands) and consist of the cause-specific number of deaths from various infectious diseases, including vaccine-preventable diseases. Deaths were stratified by year and age group (for 1903–1920: <1, 1–4, 5–13, 14–19, 20–29, 30–39, 40–49, 50–79, and ≥80 years; for 1930–1940, the same age groups were available, except that 5–14 and 15–19 replaced 5–13 and 14–19; for 1941–2012, data were available by 5-year age group, with separate groups for <1 and ≥80 years). Here we focus on the mortality due to diphtheria, pertussis, tetanus, and polio during the period 1903–1996. Data were available for the entire period, except for poliomyelitis which was included as a cause of death since 1920.
We quantified the childhood mortality burden as years of life lost before the age of 20. Each reported death was randomly assigned a specific age within each age group and a birth cohort using multiple imputation methods. Cause-specific mortality burden was then calculated using restricted mean lifetimes survival analysis.12,13 This method estimates the years of life lost due to a specific cause up to a cut-off age within a competing risks framework. For our analysis, we choose a cut-off age of 20 years, as most mortality due to our diseases of interest occurred before that age. Each 1-year birth cohort between 1903 and 1975 was followed up to 20 years of age. Cumulative incidence curves for each cause of death were constructed using the Aalen–Johansen estimator, and the corresponding age-specific childhood mortality burden attributable to each cause was calculated from the area under the cumulative incidence curves.12,13 For more details, see eAppendix section 1; https://links.lww.com/EDE/B292, which provides more detail on the data preparation and survival analysis.
Vaccination Coverage
Vaccination coverage by birth cohort was obtained from various official reports by the Dutch Health Care Inspectorate (period 1949–1969) and the Dutch National Institute for Public Health and the Environment (period 1970–1975).9 Missing cohorts (1953 for diphtheria, pertussis, and tetanus, and 1960–1961 for poliomyelitis) were linearly interpolated from adjacent birth cohorts. As far as data allowed, we used age-specific national vaccination coverage. In the early years of mass vaccination (before 1962), registration of vaccination coverage was less stringent, and it is unknown that how many vaccines each child had received at which age. For this period, vaccination coverage was determined from the number of children who had already been vaccinated at 1 year of age, and when entering kindergarten or elementary school. We assumed this calculated coverage represents the coverage at the ages of 3 months, 4 years, and 6 years of age, respectively. As the coverage metric, we used the proportion of children who had received at least one vaccine during their lifetime, and we assumed a 95% vaccine effectiveness against mortality regardless of the number of vaccine doses.14–16 In eFigures 8–11; https://links.lww.com/EDE/B292, we present sensitivity analyses exploring the impact of this choice on our results.
Mass vaccination with the diphtheria toxoid vaccine started in 1953 in the Netherlands. Before the start of the mass vaccination program against diphtheria in 1953, vaccination was already ongoing and mainly administered at 4–14 years of age.17 In 1954, the diphtheria vaccine was combined with vaccines against pertussis and tetanus (DTP). Polio vaccination followed in 1957 with a staggered catch-up campaign of all children born since 1945. The polio vaccine was combined with DTP in 1962 (DTP-IPV) and was offered at 3, 4, 5, and 11 months. Starting in 1965, revaccination with DT-IPV was offered at 4 and 9 years of age. Figure 1 shows the vaccination coverage by age (percentage of children vaccinated at least once) for each birth cohort since 1945. National vaccination coverage increased rapidly for each of these mass vaccination programs and reached 90% or higher within a decade after each vaccine introduction.9
;)
FIGURE 1: Vaccination coverage by age and birth cohort, the Netherlands, 1948–1975. Vaccination coverage in the Netherlands for (A) diphtheria; (B) pertussis; (C) poliomyelitis; and (D) tetanus. Vaccination coverage is defined as the proportion of children having received at least one dose during their lifetime.
Modeling Overview
We estimated the overall effectiveness of a vaccination program on mortality, by comparing the observed childhood mortality burden with the expected mortality burden had the vaccination program not been introduced, i.e., the counterfactual. To capture overall prevaccination trend in childhood mortality burden, the counterfactual model was based on two components: the exponential decline in all-cause childhood mortality burden and contribution of a specific disease to this all-cause childhood mortality burden. The exponential decline was modeled by fitting a linear regression model to log-transformed prevaccination all-cause childhood mortality burden over birth cohorts 1903–1940. The cause-specific contributions to the all-cause childhood mortality burden were calculated for each age separately as the ratio of the age- and cause-specific mortality burden to the total all-cause childhood mortality burden. We assumed that the age-specific contributions were constant in the prevaccination period (see eAppendix section 2; https://links.lww.com/EDE/B292, where we show that there were no relevant age trends in the prevaccination period). To reflect uncertainty, the age-specific contributions of each vaccine-preventable disease were resampled from the prevaccination period with a higher sampling weight for more recent birth cohorts. The distribution of the rate of exponential decline was obtained using the semiparametric bootstrap by resampling residuals (see eFigure 1; https://links.lww.com/EDE/B292, for the distribution of parameters used in constructing the counterfactual).
We extrapolated the counterfactual model from birth cohort 1948 up to and including the 1975 birth cohort. The overall vaccination program effectiveness in terms of mortality burden averted was defined as the ratio of observed and counterfactual childhood mortality burden. By incorporating vaccination coverage and vaccine effectiveness in the estimation, direct and indirect effects of vaccination programs can be distinguished. The direct effects of vaccination were defined as the product of the vaccination coverage and vaccine effectiveness (i.e., the expected proportion of children at a specific age who are immunized). Indirect effects were defined as the remaining childhood mortality burden averted after subtracting direct effects. The model is described in more detail in the following sections.
Counterfactual Model
We constructed the counterfactual by projecting the exponential decline forward from the start of mass vaccination programs, assuming the rate of decline in childhood mortality burden, r, and the relative contribution of cause i to the all-cause childhood mortality burden at age a, pi,a, remained constant at their prevaccination values (see eAppendix section 2; https://links.lww.com/EDE/B292). In the following, we indicate the counterfactual by superscript c = 0 and the observed situation by c = 1. Let y0 be the all-cause childhood mortality burden in birth cohort t0, then for each birth cohort with birth year t, the age-specific counterfactual childhood mortality burden was calculated as follows:
Both y0 and r were estimated by fitting a linear regression model to the log-transformed all-cause childhood mortality burden in the prevaccination period 1903–1940 (i.e., the intercept and the regression coefficient for birth cohort). The distributions of r and y0 were obtained using semiparametric bootstrap by resampling residuals. We assumed pi,a remained constant in the counterfactual situation, assuming the hypothesis that the relative contribution did not change had vaccination programs not been introduced. We estimated pi,a for each birth cohort in the prevaccination period by dividing the age-specific years of life lost due to cause i by the all-cause years of life lost. To reflect uncertainty, pi,a was resampled from the distributions in the prevaccination periods with a higher weight for more recent birth cohorts: 1903–1930 for diphtheria; 1903–1940 for pertussis; 1920–1940 for poliomyelitis; and 1903–1940 for tetanus (we excluded the Second World War).
Overall, Direct, and Indirect Program Effectiveness
The overall effectiveness of a vaccination program for cause i in birth cohort t up to age τ can be defined as the ratio of the observed mortality burden,
, and the expected mortality burden in the counterfactual,
.18
The overall program effectiveness can also be partitioned into the direct and indirect program effectiveness:
. The expected direct program effectiveness is the product of the vaccine effectiveness, v, and the vaccination coverage at age a, Ca(t), such that
. The indirect program effectiveness, defined here as any reduction in mortality burden not explained by direct protection (in other words, the difference between the observed and expected mortality burden if only direct protection would play a role), can then be calculated as follows:
We calculated the indirect effects of vaccination programs using a vaccine effectiveness against mortality of 95% for all vaccines. Varying the vaccine effectiveness had little qualitative and quantitative impact on our results other than increasing or decreasing the estimated indirect effects slightly (eFigure 8–10; https://links.lww.com/EDE/B292). Uncertainty intervals reflect the uncertainty inherent in the imputation of single-year ages from age group–specific data (Figure 2, shaded areas), combined with the resampling of prevaccination period pi,a, and bootstraps of r and y0, yielding 95% uncertainty intervals. All analyses were performed in R statistical programming environment, version 3.2.0 (R Foundation for Statistical Computing, Vienna, Austria).
FIGURE 2: Observed and estimated childhood mortality burden per live birth, the Netherlands, 1903–1975. The observed (solid), fitted (dashed), and counterfactual (dotted) (equation 1) years of life lost before the age of 20 per live birth by birth cohort in the Netherlands for (A) diphtheria; (B) pertussis; (C) poliomyelitis; and (D) tetanus. Lines indicate medians and 95% uncertainty intervals.
RESULTS
Figure 2 shows the observed childhood mortality burden along with the model fit to the prevaccination period and the estimated counterfactual (the situation had vaccination programs not been introduced). Our model adequately captures the observed prevaccination childhood mortality burden. Upon the start of mass vaccination, the counterfactual and observed childhood mortality burdens rapidly diverge. For pertussis (and to a lesser extent poliomyelitis), this divergence starts several cohorts before the start of mass vaccination. Because we look at birth cohorts, this early divergence may be due to indirect effects from vaccination of later birth cohorts or due to unregistered vaccination. This would result in a decline in mortality burden before the start of mass vaccination.
The overall effectiveness of vaccination programs against diphtheria, pertussis, poliomyelitis, and tetanus increased rapidly after the start of mass vaccinations and reached near 100% within 10 birth cohorts for each vaccine-preventable disease (Figure 3).
FIGURE 3: Overall vaccination program effectiveness and vaccination coverage, the Netherlands, 1903–1975. Overall program effectiveness and vaccination coverage in the Netherlands for (A) diphtheria; (B) pertussis; (C) poliomyelitis; and (D) tetanus. Lines indicate medians and 95% uncertainty intervals for the prevaccination period (dashed) and the vaccination period (solid). Overall program effectiveness is defined as the ratio of averted childhood mortality burden to the expected childhood mortality burden had vaccination programs not been introduced (equation 2). The dashed lines represent the same calculation carried over to the prevaccination as a control. Vaccination coverage at 1 year of age is indicated by the diamond symbols, where open symbols represent interpolated data points. Here we assume a vaccine effectiveness against mortality of 95%.
Figure 4 shows the estimated direct and indirect vaccination program effectiveness in the Netherlands up to and including the 1975 cohort for diphtheria, pertussis, poliomyelitis, and tetanus. Mainly diphtheria and pertussis showed signs of indirect protection, with a maximum estimated indirect effect of 0.25 (95% uncertainty interval [UI] = 0.24, 0.25) in birth cohort 1960 for diphtheria, and 0.62 (UI = 0.54, 0.69) in birth cohort 1951 for pertussis. Over time, as the proportion of children who got vaccinated increased and the direct program effectiveness increased, the indirect program effectiveness for diphtheria and pertussis diminished. We expected to see indirect effects for poliomyelitis, but there was no clear evidence of indirect effects as in most birth cohorts, the uncertainty intervals are broad and overlap with zero. No indirect effects were observed for tetanus, which is to be expected as it is not a communicable disease. By birth cohort 1965, almost no mortality burden was observed due to pertussis, diphtheria, poliomyelitis, or tetanus and the indirect program effectiveness is reduced to the complement of the direct program effectiveness (equation 3); hence, the small, positive value seen in all plots between the 1965 and 1975 birth cohorts.
FIGURE 4: Direct and indirect vaccination program effectiveness, the Netherlands, 1948–1975. Direct (dashed) and indirect (solid) effectiveness of vaccination programs in the Netherlands for (A) diphtheria; (B) pertussis; (C) poliomyelitis; and (D) tetanus. Lines indicate medians and 95% uncertainty intervals. Note that direct and indirect program effectiveness sum to the overall program effectiveness. Here we assume a vaccine effectiveness against mortality of 95%.
Overall, since birth cohort 1948 and up to the 1975 birth cohort, 14.9% (95% UI = 12.3%, 17.6%) of all childhood mortality burden averted due to diphtheria vaccination was due to indirect protection (Figure 4 and eFigure 2; https://links.lww.com/EDE/B292, which shows the averted mortality burden due to indirect effects). For pertussis, 32.1% (95% UI = 31.3%, 32.8%) was due to indirect protection. For poliomyelitis and tetanus, this was −2.4% (UI = −16.7%, 7.1%) and 0.6% (UI = −17.9%, 10.7%), respectively.
DISCUSSION
According to our analysis, there are substantial indirect effects of mass vaccination against diphtheria and pertussis on childhood mortality burden, and program effectiveness was considerably higher than would be expected based on direct effects of vaccination alone. These indirect effects were especially high at the start of mass vaccination when vaccine coverage was still low; up to 25% of the averted diphtheria mortality burden was due to indirect effects and up to 62% for pertussis. These results provide evidence for herd protection, of which the impact seems to be highest in the early years of vaccination programs when vaccination coverages (and direct effects) were still relatively low.
We did not observe indirect effects due to vaccination against poliomyelitis. This was unexpected and may be due to the low number of observed deaths or due to the regular epidemics in the prevaccination period, which increased the uncertainty in our analysis. The staggered catch-up campaign of all children 15 years old and younger initiated in 1957, together with the broad age distribution of deaths due to poliomyelitis, further increased the width of the uncertainty intervals for both direct and indirect program effectiveness. Although we did estimate a high overall effectiveness of vaccination programs (Figure 3), our method may not be sensitive enough to detect indirect effects for poliomyelitis. It is likely that any indirect effects are more apparent in morbidity data than mortality. Here we restricted our analysis to mortality data, as these detailed data have been systematically collected for long-time periods. Similarly, indirect effects for pertussis and diphtheria, although present in mortality statistics, may be more pronounced in morbidity data.
We included tetanus, a noncommunicable vaccine-preventable disease, as a negative control for which we expect no indirect protection; indeed, we observed no indirect effects. To further check our calculation, we verified that the overall program effectiveness (equation 2) over the prevaccination period was zero (Figure 3). If our model performs well, the “overall program effectiveness” in the prevaccination period should fluctuate around the null and rapidly increase from the start of mass vaccination. This was the case for each vaccine-preventable disease in our study period (a decline can be seen for diphtheria during the Second World War when large epidemics swept across Europe19). This finding, together with the near-zero estimate of indirect protection for the tetanus vaccination program, gives credence to our methodology.
There are several limitations and possible biases to our approach. First of all, our estimated counterfactual may be overestimated due to other, unaccounted for, factors unrelated to vaccination that also impact upon childhood mortality burden, most notably the increasing use of antibiotics around the time mass vaccination programs started. The impact of antibiotics is partially taken into account by the exponential decline in the all-cause childhood mortality burden, but may still show residual impact. This would lead to an overestimation of our indirect effects. To investigate the potential impact of antibiotics on our results, we performed additional analyses (see eAppendix section 3; https://links.lww.com/EDE/B292, for details on the sensitivity analysis) in which we assume that antibiotics reduce the mortality burden in the counterfactual either by a constant or by increasing the rate of exponential decline. These analyses indicated that our results are sensitive to the influence of antibiotics, specifically if they influence mortality burden by increasing the exponential decline. However, even at a moderately high impact of antibiotics, indirect effects are still present for pertussis and diphtheria. These effects decrease rapidly as the effect of antibiotics increases. Although our results are influenced by the potential reduction in mortality burden due to antibiotics (and other prevention measures than vaccination), if their impact is limited, indirect effects remain apparent, especially at lower levels of vaccination coverage.
Second, we assumed that the contributions of vaccine-preventable diseases to the total mortality burden remained constant in the counterfactual. This is a reasonable assumption for most vaccine-preventable diseases, given their small and relatively constant contributions to the total mortality burden in the prevaccination period.9 The constancy assumption is also attractive as one would expect the cause-specific mortality burden to decline at a similar rate to the total mortality burden. In additional analyses, we tested whether there were any age-specific trends in the prevaccination period (see eAppendix section 2; https://links.lww.com/EDE/B292). We did not find significant trends in the prevaccination period; however, any effects of existing trends would be overwhelmed by the existing uncertainty in the analysis. Our assumption of a constant contribution from all vaccine-preventable diseases to the counterfactual seems justified, and allows us to restrict the model to the decline in all-cause mortality, obviating the need to construct multiple disease-specific models.
Third, we assumed that the exponential decline in the prevaccination period would hold subsequent to the start of mass vaccination. Exponential declines in (childhood) mortality rates throughout the 20th century have been observed in many developed countries besides the Netherlands.20 For our timeframe of interest—the period directly following the introduction of mass vaccinations—it is unlikely that the trends in the counterfactual would look radically different than those in the prevaccination period, as these have been stable for the entire prevaccination period.
Another factor that may have biased our results is the uncertainty in registered vaccination coverage. Registration of vaccination coverage improved over time, and starting 1962, detailed records are available. In the early years of mass vaccination, registration of vaccination status was often incomplete, and before the implementation of nationwide mass vaccination programs, there may have been substantial unregistered vaccination taking place.17 Our vaccination coverage data may thus underestimate actual coverage. In addition, it is unclear how many children and at what age children were vaccinated; a substantial number of vaccines may have been given to children who were already immune due to natural infection. Our assumed vaccine effectiveness of 95% may therefore be unrealistic in the early part of the vaccination period. Decreasing vaccine effectiveness slightly increased the indirect effects, and increasing vaccine effectiveness slightly decreased the indirect effects (eFigures 8–11; https://links.lww.com/EDE/B292). The interplay of these factors makes it difficult to determine if the vaccination coverage—and in extension the proportion immunized and our estimated indirect effects—is biased and in which direction.
This research focused on the population-wide direct and indirect effectiveness of vaccination programs on mortality in the Netherlands. Indirect protection is a well-established phenomenon in infectious disease epidemiology.7 However, a quantitative estimate of the magnitude of indirect protection compared with direct protection has been lacking for the older vaccination programs, and specifically, the literature on the population effectiveness of vaccination programs using population-wide surveillance data is deficient.21 Other studies looking into the population effectiveness of vaccination programs have mainly focused on contemporary vaccines such as meningococcal serogroup C conjugate vaccines,22–24 multivalent pneumococcal conjugate vaccines,25,26 rotavirus vaccines,27,28Haemophilus influenza type b conjugate vaccines,29,30 and influenza vaccines.31 Our research provides a quantitative insight into the population direct and indirect effectiveness of older vaccination programs using already existing data sources.
Future research should focus on the effectiveness of vaccination programs on morbidity by including hospitalization or notification data. This is especially important for diseases such as poliomyelitis for which program effectiveness may not be well estimated using mortality data, and for which a major share of disease burden is attributed to long-term sequelae. Alternatively, our methods could be verified using mortality data from other countries. In addition, spatial heterogeneity should be accounted for, as vaccination coverage shows substantial geographical differences.32 This heterogeneity may provide more insight into the indirect effects of vaccination when comparing high- and low-coverage regions.
Our analysis shows that the indirect effects of the early vaccination programs for diphtheria and pertussis are pronounced even in mortality statistics, indicating that for a proper appreciation of the impact of vaccination programs and the monitoring of their effectiveness, both direct and indirect effects should be taken into account.
ACKNOWLEDGMENTS
We thank Statistics Netherlands for providing access to the data used in this study.
REFERENCES
1. Metcalf CJ, Ferrari M, Graham AL, Grenfell BTUnderstanding herd immunity. Trends Immunol. 2015;36:753–755.
2. Schuchat A, Bell BPMonitoring the impact of vaccines postlicensure: new challenges, new opportunities. Expert Rev Vaccines. 2008;7:437–456.
3. Halloran ME, Struchiner CJStudy designs for dependent happenings. Epidemiology. 1991;2:331–338.
4. Shim E, Galvani APDistinguishing vaccine efficacy and effectiveness. Vaccine. 2012;30:6700–6705.
5. Haber MEstimation of the population effectiveness of vaccination. Stat Med. 1997;16:601–610.
6. Haber MEstimation of the direct and indirect effects of vaccination. Stat Med. 1999;18:2101–2109.
7. Fine PEHerd immunity: history, theory, practice. Epidemiol Rev. 1993;15:265–302.
8. Taranger J, Trollfors B, Bergfors E, et alMass vaccination of children with pertussis toxoid–decreased incidence in both vaccinated and nonvaccinated persons. Clin Infect Dis. 2001;33:1004–1010.
9. van Wijhe M, McDonald SA, de Melker HE, Postma MJ, Wallinga JEffect of vaccination programmes on mortality burden among children and young adults in the Netherlands during the 20th century: a historical analysis. Lancet Infect Dis. 2016;16:592–598.
10. Wolleswinkel-van den Bosch JH, Looman CW, Van Poppel FW, Mackenbach JPCause-specific mortality trends in the Netherlands, 1875-1992: a formal analysis of the epidemiologic transition. Int J Epidemiol. 1997;26:772–781.
11. Lipsitch M, Jha A, Simonsen LObservational studies and the difficult quest for causality: lessons from vaccine effectiveness and impact studies. Int J Epidemiol. 2016;45:2060–2074.
12. Andersen PKDecomposition of number of life years lost according to causes of death. Stat Med. 2013;32:5278–5285.
13. Andersen PK, Canudas-Romo V, Keiding NCause-specific measure of life years lost. Demogr Res. 2013;29:1127–1152.
14. Hamborsky J, Kroger A, Wolfe SEpidemiology and Prevention of Vaccine-Preventable Diseases. 2015.13th ed. Washington, DC: Public Health Foundation;
15. Bisgard KM, Rhodes P, Hardy IR, et alDiphtheria toxoid vaccine effectiveness: a case-control study in Russia. J Infect Dis. 2000;181(suppl 1):S184–S187.
16. Plotkin SA, Orenstein WA, Offit PAVaccines. 2013.6th ed. Philadelphia, PA: Elsevier Saunders;
17. Hoogendoorn DDe inentingen tegen diphtherie en de propaganda hiervoor [Immunizations against diphtheria and its propaganda]. Ned Tijdschr Geneeskd. 1954;98:1806–1809.
18. Halloran ME, Struchiner CJ, Longini IM JrStudy designs for evaluating different efficacy and effectiveness aspects of vaccines. Am J Epidemiol. 1997;146:789–803.
19. Stuart GDiphtheria incidence in European countries. Br Med J. 1945;2:613–615.
20. Tuljapurkar S, Li N, Boe CA universal pattern of mortality decline in the G7 countries. Nature. 2000;405:789–792.
21. Breiman RF, Streatfield PK, Phelan M, Shifa N, Rashid M, Yunus MEffect of infant immunisation on childhood mortality in rural Bangladesh: analysis of health and demographic surveillance data. Lancet. 2004;364:2204–2211.
22. Trotter CL, Andrews NJ, Kaczmarski EB, Miller E, Ramsay MEEffectiveness of meningococcal serogroup C conjugate vaccine 4 years after introduction. Lancet. 2004;364:365–367.
23. Ramsay ME, Andrews NJ, Trotter CL, Kaczmarski EB, Miller EHerd immunity from meningococcal serogroup C conjugate vaccination in England: database analysis. BMJ. 2003;326:365–366.
24. Bijlsma MW, Brouwer MC, Spanjaard L, van de Beek D, van der Ende AA decade of herd protection after introduction of meningococcal serogroup C conjugate vaccination. Clin Infect Dis. 2014;59:1216–1221.
25. Grijalva CG, Griffin MRPopulation-based impact of routine infant immunization with pneumococcal conjugate vaccine in the USA. Expert Rev Vaccines. 2008;7:83–95.
26. Jokinen J, Rinta-Kokko H, Siira L, et alImpact of ten-valent pneumococcal conjugate vaccination on invasive pneumococcal disease in Finnish children—a population-based study. PLoS One. 2015;10:e0120290.
27. Panozzo CA, Becker-Dreps S, Pate V, et alDirect, indirect, total, and overall effectiveness of the rotavirus vaccines for the prevention of gastroenteritis hospitalizations in privately insured US children, 2007-2010. Am J Epidemiol. 2014;179:895–909.
28. Pollard SL, Malpica-Llanos T, Friberg IK, Fischer-Walker C, Ashraf S, Walker NEstimating the herd immunity effect of rotavirus vaccine. Vaccine. 2015;33:3795–3800.
29. O’Loughlin RE, Edmond K, Mangtani P, et alMethodology and measurement of the effectiveness of
Haemophilus influenzae type b vaccine: systematic review. Vaccine. 2010;28:6128–6136.
30. Morris SK, Moss WJ, Halsey N
Haemophilus influenzae type b conjugate vaccine use and effectiveness. Lancet Infect Dis. 2008;8:435–443.
31. Baguelin M, Flasche S, Camacho A, Demiris N, Miller E, Edmunds WJAssessing optimal target populations for influenza vaccination programmes: an evidence synthesis and modelling study. PLoS Med. 2013;10:e1001527.
32. van Lier E, Oomen P, Conyn-van Spaendonck M, Drijfhout I, Zonnenberg-Hoff I, de Melker HVaccinatiegraad Rijksvaccinatieprogramma Nederland [Vaccincation Coverage National Immunization Programme Netherlands]. 2015.Bilthoven, the Netherlands: National Institute for Public Health and the Environment;
