The SAGE review put a major emphasis on assessing bias in the studies included in the review.6 The criteria for assessment are discussed in the Appendix. Ten studies were considered to have “high risk of bias.” Six studies (marked in grey in Table 1) had “very high risk of bias” and were not considered in the main analysis of the effect of DTP-vaccinated versus DTP-unvaccinated.
The assessment did not take the direction of bias into consideration nor did it examine whether the potential biases had the predicted effect (see below). It has no meaning to exclude a study because of “very high risk of bias” if it shows a negative effect of DTP and all known biases should lead to a positive estimate for DTP. We have therefore initially maintained all 16 studies in the analysis and explored how the biases may affect the direction of the estimated effect of DTP.
Bias Increasing the Benefit of Vaccination
Frailty bias and survival bias are the most important biases exaggerating the benefit of vaccination.
If it is the healthy children with the most educated and compliant mothers, living close to a health facility, who are brought first for vaccination, this would imply that the vaccinated children have inherent lower risk of dying6; in other words, mortality is “unnaturally” high in the “unvaccinated” group and estimated benefits of vaccination may be exaggerated.
We reviewed the 16 studies in SAGE review for indication of whether healthy children were vaccinated first (Table 3).
Most of the existing DTP studies have been conducted in health and demographic surveillance system sites, where information on vaccination status and vital status is collected at home visits with certain intervals (Fig. 1). Retrospective updating of vaccine status for survivors but not for children who died lead to survival bias; because of lack of information during follow-up dead children will be classified as unvaccinated even though they might have been vaccinated (Fig. 1, scenario 1).
Misclassification of dead children may also occur when children are classified as “unvaccinated” because no vaccination card was seen; some children may have been DTP-vaccinated but this is not known to the investigators because the child died before the information could be retrieved (Fig. 1, scenario 2). Many DTP studies included in the SAGE review assumed that they had all relevant information and excluded no children16; hence, dead children with no recorded information about DTP were automatically classified as “DTP-unvaccinated.”
If the children have not been seen as unvaccinated in the age group being examined, imprecision in the day-of-birth and the day-of-death may easily lead to misclassified deaths in the unvaccinated group. In demographic surveillance systems, dates may not be known very precisely, and the day may just be classified as “15” and even the month may be uncertain. With this lack of precision, a child who biologically dies at 35 days of age, before DTP-vaccination is given at 6 weeks of age, may end up being classified as having died at 45 days as a DTP-unvaccinated child. We have called these inadequacies for a “poorly defined control group.” ie, the group of “unvaccinated” has not been defined actively by determining that the children were “unvaccinated” and alive at the relevant age.
Bias Reducing the Benefit of Vaccination
Other than publication bias,6 3 biases have been suggested to lead to “unnatural” high mortality in the vaccinated groups.6,27
In contrast to the speculation above that healthy children are vaccinated first, it has also been suggested that sick children might come to health clinics and get preferential vaccination, creating a bias which would reduce the apparent benefits of vaccination.27 As discussed below, the data does not make that possibility very likely (Table 3).
Starting observation time sometime after the actual vaccination occurred (as would happen when data is collected in a demographic surveillance system with regular data collection rounds and a “landmark approach”) could mean that frail children had already died in the unvaccinated group, leading to higher measured mortality in the DTP group.6 It has been shown specifically in one study that the delay in starting follow-up had no impact on the estimate25 and several studies have started follow-up at the day of vaccination24,26 and found similar strong negative effects so this bias is unlikely to be important.
Selective Censoring of Children Vaccinated During Follow-up
Censoring for measles vaccination during follow-up could mean that the healthiest DTP-vaccinated children had gone on to be measles vaccinated and frail children were left to die in the DTP group.6 The studies which have tested whether censoring for MV matters for the estimated DTP effect have found no effect or the opposite effect19,32 and many studies have not censored for MV and still found a strong negative effect of DTP8,11,24,29 so this bias is unlikely to be important.
Frailty bias and different forms of survival bias (Fig. 1) will lead to an unnaturally high mortality rate in the unvaccinated group.15–17 To measure this bias, we have defined a “bias index” as the mortality rate ratio (MRR) comparing children classified as completely unvaccinated versus children classified as having received at least one vaccine. We calculated the bias index for the 14 of the 16 SAGE studies in which the necessary information was available (Fig. 2 and Table 1).
Direction of Bias
The direction of bias is just as important as the inclusion or exclusion of studies because of bias. For the sake of determining whether DTP is associated with increased mortality, based on a majority of studies showing mortality estimates above 1 when comparing DTP-vaccinated children versus DTP-unvaccinated children, the important question is whether a study has biases which would lead to exaggeration of the harm from DTP. Such studies should be excluded or at least very carefully considered. In contrast, among studies, which find an estimate above 1, it should not be a reason to exclude studies that have biases which leads to underestimation of the harm from DTP.
Available data on links between nutritional status and vaccination are summarized in Table 3. Three studies reported that sick children were not vaccinated. Because WHO has previously recommended to vaccinate sick children, one study speculated that sick children might have been more likely to get vaccinated, but presented no data.27 Six studies reported nutritional status at time of vaccination or nutritional status and subsequent time to vaccination; all studies except one suggested that healthier children were vaccinated first. Hence, most studies suggest that frail children are less likely getting vaccinated, and we should expect that the DTP-unvaccinated children inherently have higher mortality than the DTP-vaccinated children.
DTP Studies and the Bias Index
The 14 studies for which a bias index could be calculated are presented in Figure 2. Eight studies had prospective follow-up and no survival bias8,24–30 (Table 1); in 7 of these, a bias index could be calculated, being between 0.49 and 1.74. Two further studies had limited survival bias.11,23 In Senegal,23 the “unvaccinated” children could not be actively identified from the way the data had been collected; however, vaccination coverage was very low23 so a few misclassified children in the unvaccinated group had no effect on the bias index which was only 1.48 (1.23–1.77). In India-II,11 it was not described that “unvaccinated” was documented; however, the children were visited every 2 weeks so most vaccinations would have been registered and the bias index was 1.64 (0.87–3.07). Inclusion or exclusion of these 2 studies made no difference to the overall results.
Of the remaining 6 studies, one had mainly given BCG and DTP simultaneously22 and therefore did not evaluate the effect of DTP after BCG. The last 5 studies had a poorly defined control group and a bias index between 2.0 and 8.0 (Fig. 2 and Table 1).9,12,13,20,21
The different focus in the SAGE and our assessment of bias give rise to 6 discordant study classifications. The SAGE review included 3 studies, which we believe should not have been included because of massive frailty and survival bias, and excluded 3 studies which should have been included, because they had limited bias or the risk estimate was above 1 and the direction of the bias was away from harm (towards 1).
First, the SAGE-review's main analysis of DTP included 3 studies from Papua New Guinea (PNG), Bangladesh and Burkina Faso, which all had high bias indexes. The PNG had a 1–5 months mortality rate of 233 per 1000 person-years in unvaccinated children compared with only 31 in vaccinated children.9 A rate of 233/1000 is unrealistic in a community with a neonatal mortality rate of 18 per 1000, and a general postneonatal mortality rate of only 48 per 1000 person-years. The rate is also higher than in 1920s before vaccine and antibiotics were introduced.33 A main reason may have been that sick children were not vaccinated9 and that the study had various forms of survival bias. The Bangladesh study19 is based on community registers and it was assumed that all information was available because no child was excluded for lack of information; thus, status as “unvaccinated” was not actively verified. The study had problems with registration of early vaccination events; children who subsequently moved had significantly lower vaccination coverage. Seventy per cents of the children started with BCG+DTP1-first. Controlling for the relevant background factors, children who received DTP1 after BCG as currently recommended by WHO had a MRR of 1.78 (1.03–3.06) compared with children who had received BCG and DTP1 simultaneously.19 The SAGE reviewers used the DTP1 group from the BCG-first arm to measure the effect of DTP after BCG; however, if early events are not registered some DTP1-vaccinated children who died may not have been registered and have ended up as misclassified deaths in the BCG-group to produce a MRR of 0.52 (0.31–0.87) for DTP1 after BCG (Table 3). Because the DTP1-after-BCG group had significantly higher mortality than children who had received BCG and DTP1 simultaneously, it seems strange to use this study to suggest that DTP after BCG has a good effect on child survival.19 The Burkina Faso study12 assumed that children not seen were unvaccinated which would lead to survival bias. An undocumented proportion of the children had received DTP with BCG because the median age of BCG vaccination was 4.8 months, or received MV during follow-up, and both combinations would reduce any harm from DTP. Thus, the design of these 3 studies, which suggested benefit or no harm of DTP, implied a risk of bias that may have made DTP appear more beneficial than it was.
Second, the SAGE-review's main analysis excluded 3 studies, Ghana-II, Guinea-Bissau IV and India-III, with bias indexes of 0.67–1.74 (Table 1) for having a “very high risk of bias.”6 Ghana-II28 was classified as very high risk of bias because of delayed entry after DTP vaccination and because a high proportion had coadministration of BCG or MV; both of these vaccines reduce the apparent harm from DTP reported by the study so it should not be a reason to exclude the study. According to the SAGE review Guinea-Bissau-IV29 had delayed entry after DTP and children who received MV were excluded.6 The study measured the effect of pre-war vaccination status during a civil war where there were few vaccinations29; to measure the effect of DTP, the children who had already received MV were evidently not included in the analysis of DTP. Hence, the reasons for excluding the study are unclear. India-III30 was excluded as “very high risk of bias” because of an unadjusted comparison of older DTP-vaccinated children with younger DTP-unvaccinated children and coadministration of MV. Both factors would reduce the apparent harm from DTP reported by the study and are not a reason to exclude. Furthermore, the paper did actually present another age and weight-adjusted estimate of 1.36 (0.63–2.92) which censored at 9 months to eliminate the effects of coadministration of MV. Thus, the 3 studies, which suggested harm of DTP, were excluded for reasons that had little or no effect on the estimate, or would reduce the apparent harm from DTP.
DTP and Child Survival
Based on the MRRs for DTP-vaccinated versus DTP-unvaccinated children presented in Table 1 (last column), the executive summary of the SAGE review concluded that findings for DTP were inconsistent, with a majority of the studies indicating a detrimental effect of DTP and 2 studies (Bangladesh, PNG) indicating a beneficial effect.6 In the ten studies, in the SAGE analysis, the MRR for DTP-vaccinated versus DTP-unvaccinated children was 1.38 (0.92–2.08).
In Our Assessment
In the 5 studies with major survival and frailty bias (bias index above 2; Fig. 2), there was no or a beneficial effect of DTP, the meta-estimate being 0.39 (0.18–0.83) (random effect model; Table 1).
In contrast, in the 8 studies in which “unvaccinated” was assessed by vaccination card or health center records, the MRR for DTP-vaccinated versus DTP-unvaccinated children was 2.00 (1.50–2.67; Fig. 3). Including the 2 studies with minimal survival bias did not change the result, the MRR being 1.89 (1.49–2.43).11,23 It made no difference if the 3 studies judged by the SAGE reviewers to have “very high risk of bias” were excluded [MRR = 1.91 (1.46–2.50)].
Mortality After Other Vaccines in the Same Population
Given that the healthiest children are most likely to be vaccinated first (Table 3) no bias should be able to produce the counterintuitive trend of 2-fold higher mortality for DTP (Fig. 3). Had there been an unrecognized bias producing higher mortality for vaccinated children, this bias would presumably also have resulted in higher mortality for BCG and measles vaccinated children. As seen in Figure 4, in the studies, which estimated the effect of several vaccines, DTP was consistently associated with higher mortality whereas BCG and MV were associated with lower mortality; for studies estimating the effect of BCG the reduction was 44% (32%–54%) and for MV 54% (40%–65%).
In 2004, GACVS reviewed the studies on DTP and concluded that a deleterious effect of DTP on child survival was not supported by the evidence.14 That conclusion was because of the inclusion of studies with survival bias15,16,34; furthermore, most studies had given BCG and DTP simultaneous and not BCG at birth and DTP 6 weeks later as recommended by WHO. In the recent SAGE review, the studies with simultaneous BCG and DTP vaccinations have mostly been excluded because it was acknowledged that BCG and DTP simultaneously may have quite different effects from BCG followed by DTP.23,30 Furthermore, many of the studies with survival bias were excluded as it was recognized that they had very high risk of bias.
The SAGE review did not dismiss a possible deleterious effect of DTP; however, evoking the high risk of bias in all studies and inconsistent results, SAGE concluded that the available data neither excluded nor confirmed the possibility of beneficial or deleterious NSEs of DTP on all-cause mortality6; however, the data presented in Figure 2 (Table 1) suggest that the estimated effect of DTP-vaccination versus no DTP-vaccination is only beneficial when the bias index is very high because the mortality rate in the unvaccinated group is unnaturally high. If children for whom no information on vaccination is available are assumed to be “unvaccinated” one will automatically get a very good estimate for DTP-vaccinated compared with DTP-unvaccinated. On the other hand, when the status as “unvaccinated” has been documented, DTP was associated with a marked deleterious effect.
The SAGE review has apparently overlooked the extent and implications of survival and frailty bias. In spite of previous reviews of the methodologies and analyses of NSEs of vaccine15–17,34,35 suggesting that we should focus less on identifying presence of bias—which is inevitable in observational studies—and more on assessing the likely impact on the results,17 the SAGE review included several studies with survival bias.
An illustration of the importance of survival bias stems from our initial DTP study.8 In this study, we used the “landmark approach” and only included prospective follow-up, the bias index was 1.35 (0.97–1.89) and the estimate for DTP vaccinated versus DTP-unvaccinated was 1.84 (1.10–3.10)8; however, if we updated information on vaccinations retrospectively in the same data set, thereby introduced survival bias and a higher mortality rate in the unvaccinated group (Table 1), the bias index became 2.96 (2.15–4.08) and now DTP had a major beneficial effect on child survival [MRR = 0.62 (0.41–0.92)].31
There is no absolute value for when the bias index is unnaturally high but this example31 and the data presented in Figure 2 suggest that when the bias index is above 2 there is reason to be skeptical. By including studies with a “poor definition of the control group” and a consequent high bias index, the SAGE review has produced inconsistent results for DTP.6
The studies in Tables 1 and 2 without survival bias were not all ideal. In several studies, children who initially were DTP-unvaccinated may have received DTP during follow-up; that bias would produce a conservative estimate and not exaggerate the effect of DTP. Because the negative effect of DTP has been ascribed to an effect specific to Guinea-Bissau,14 it should be noted that 3 quarters of the studies in Table 1 were from other countries.
The interest in NSEs of vaccine was originally provoked by trials of high-titer measles vaccines (HTMV) conducted in the 1980s which found that early administration of HTMV at 4 months of age was effective against measles infection, but surprisingly associated with 2-fold higher female mortality than standard MV delivered from 9 months of age (no difference for males).7 WHO withdrew HTMV in 1992.7 Both types of MV were fully protective against measles infection, so this was a nonspecific effect. HTMV was administered at 4–5 months of age and most children received DTP or inactivated polio vaccine after HTMV. When we reanalyzed the HTMV studies from that perspective, the increased female mortality in the HTMV group was limited to girls who had received DTP/inactivated polio vaccine after HTMV but there was no increase in female mortality when HTMV was the most recent vaccination.7 Hence, it was having DTP as the most recent vaccine rather than HTMV per se, which had caused excess female mortality.
In 2000, we first reported 84% (10%–210%) higher mortality associated with DTP vaccination.8 Now many studies later, the combined estimate for studies with a comparable methodology is a 2-fold increase in mortality. Not a single study without frailty bias and with prospective follow-up has shown DTP to be associated with a beneficial effect on child survival.
There are numerous implications of recognizing that DTP may not be the best vaccine for child survival. We need to study both the underlying mechanisms and possible ways of preventing these negative effects. Given imprecision of date-of-death in most low-income country studies it has not been possible to determine whether there is a specific timing to the excess deaths after DTP. The increased mortality rate may also relate to general changes in immune profile which affect susceptibility whenever the child is exposed to other infections4,5; in that scenario a clear timing pattern of deaths would not be expected. A negative effect of DTP could be minimized by following a live-vaccine-last policy and giving a live vaccine shortly after the last DTP2; for example, we have shown that an early MV shortly after DTP3 reduced overall mortality.36 Following the same principle DTP should not be given with or after MV as this is always associated with increased mortality,2 as also recognized in the SAGE review.6 Several studies have suggested that coadministration of BCG and DTP reduces the negative effect of DTP for girls.23,25,30 Hence, there are reasons to explore whether there are other ways of reducing the DTP associated excess mortality. Furthermore, a live pertussis vaccine is being developed and apparently has beneficial NSEs in animal models.37
The global health community has used the coverage for DTP3 as the main performance indicator for the global immunization program.38 This has led to increases in the DTP coverage but much less emphasis has been placed on the timeliness and coverage of BCG and MV, the vaccines associated with lower mortality. This may need to change.
SAGE recommended that the Immunization and Vaccine related Implementation Research Advisory Committee (IVIR-AC) should prioritize research questions on the NSEs of vaccines to inform policy. IVIR-AC has decided to guide the development of standard protocols and implementation of high quality prospective studies,39 but has also recently asserted that “the impact of DTP on all-cause mortality could not be determined.”40 This seems to be minimizing the unpleasant conclusion in the SAGE review that the majority of studies suggested a deleterious effect of DTP.6 Future randomized trials could examine the unbiased effect of booster DTP or of coadministering BCG and DTP. To the extent further observational studies are conducted it is important to prevent frailty and survival bias and to recognize that commonly considered biases cannot produce a negative effect of DTP.
1. Meeting of the Strategic Advisory Group of Experts on immunization, April 2014 - conclusions and recommendations. Weekly Epidemiological Rec 2014:89:221–236.
2. Aaby P, Benn C, Nielsen J, et al. Testing the hypothesis that diphtheria-tetanus-pertussis vaccine
has negative non-specific and sex-differential effects on child survival in high-mortality countries. BMJ Open. 2012;2:e000707.
3. Aaby P, Whittle H, Benn CS. Vaccine programmes must consider their effect on general resistance. BMJ. 2012;344:e3769.
4. Benn CS, Netea MG, Selin LK, et al. A small jab – a big effect: nonspecific immunomodulation by vaccines. Trends Immunol. 2013;34:431–439.
5. Kleinnijenhuis J, Quintin J, Preijers F, et al. Bacille Calmette-Guerin induces NOD2-dependent nonspecific protection from reinfection via epigenetic reprogramming of monocytes. Proc Natl Acad Sci U S A. 2012;109:17537–17542.
7. Aaby P, Jensen H, Samb B, et al. Differences in female-male mortality after high-titre measles vaccine and association with subsequent vaccination with diphtheria-tetanus-pertussis
and inactivated poliovirus: reanalysis of West African studies. Lancet. 2003;361:2183–2188.
8. Kristensen I, Aaby P, Jensen H. Routine vaccinations and child survival: follow up study in Guinea-Bissau, West Africa. BMJ. 2000;321:1435–1438.
9. Lehmann D, Vail J, Firth MJ, de Klerk NH, Alpers MP. Benefits of routine immunisations on childhood survival in Tari, Southern Highlands Province, Papua New Guinea. Int J Epidemiol 2005;34:138–148.
10. Elguero E, Simondon KB, Vaugelade J, et al. Non-specific effects of vaccination on child survival? A prospective study in Senegal. Trop Med Int Health. 2005;10:956–960.
11. Moulton LH, Rahmathullah L, Halsey NA, et al. Evaluation of non-specific effects of infant immunizations on early infant mortality in a southern Indian population. Trop Med Int Health. 2005;10:947–955.
12. Vaugelade J, Pinchinat S, Guielle G, et al. Lower mortality in vaccinated children: follow up study in Burkina Faso. BMJ. 2004;329:1309–1311.
13. Breiman RF, Streatfield PK, Phelan M, et al. Effect of infant immunisation on childhood mortality in rural Bangladesh: analysis of health and demographic surveillance data. Lancet. 2004;364:2204–2211.
15. Fine PE, Smith PG. ‘Non-specific effects of vaccines’ – an important analytical insight, and call for a workshop. Trop Med Int Health. 2007;12:1–4.
16. Jensen H, Benn CS, Lisse IM, et al. Survival bias
in observational studies of the impact of routine immunizations on childhood survival. Trop Med Int Health. 2007;12:5–14.
17. Farrington CP, Firth MJ, Moulton LH, et al. Epidemiological studies of the non-specific effects of vaccines: II–methodological issues in the design and analysis of cohort studies. Trop Med Int Health. 2009;14:977–985.
18. Meeting of Global Advisory Committee on Vaccine Safety, 18–19 June 2008. Week Epid Rec. 2008;83:287–292.
19. Aaby P, Ravn H, Andersen A. Combined BCG and DTP vaccinations may reduce infant mortality more than the WHO-schedule of BCG first and then DTP. A re-analysis of demographic surveillance data from rural Bangladesh. Draft manuscript provided to the SAGE review committee.
20. Krishnan A, Srivastava R, Dwivedi P, et al. Non-specific sex-differential effect of DTP vaccination may partially explain the excess girl child mortality in Ballabgarh, India. Trop Med Int Health. 2013;18:1329–1337.
21. Bawah AA, Phillips JF, Adjuik M, et al. The impact of immunization on the association between poverty and child survival: evidence from Kassena-Nankana District of northern Ghana. Scand J Public Health. 2010;38:95–103.
22. Chan GJ, Moulton LH, Becker S, et al. Non-specific effects of diphtheria tetanus pertussis vaccination on child mortality in Cebu, The Philippines. Int J Epidemiol. 2007;36:1022–1029.
23. Aaby P, Nielsen J, Benn CS, et al. Sex-differential and non-specific effects of routine vaccinations in a rural area with low vaccination coverage: an observational study from Senegal. Trans R Soc Trop Med Hyg. 2015;109:77–84.
24. Aaby P, Jensen H, Gomes J, et al. The introduction of diphtheria-tetanus-pertussis vaccine
and child mortality in rural Guinea-Bissau: an observational study. Int J Epidemiol. 2004;33:374–380.
25. Aaby P, Ravn H, Roth A, et al. Early diphtheria-tetanus-pertussis
vaccination associated with higher female mortality and no difference in male mortality in a cohort of low birthweight children: an observational study within a randomised trial. Arch Dis Child. 2012;97:685–691.
26. Aaby P, Vessari H, Nielsen J, et al. Sex differential effects of routine immunizations and childhood survival in rural Malawi. Pediatr Infect Dis J. 2006;25:721–727.
27. Velema JP, Alihonou EM, Gandaho T, et al. Childhood mortality among users and non-users of primary health care in a rural west African community. Int J Epidemiol. 1991;20:474–479.
28. Welaga P, Nielsen J, Adjuik M, et al. Non-specific effects of diphtheria-tetanus-pertussis
and measles vaccinations? An analysis of surveillance data from Navrongo, Ghana. Trop Med Int Health. 2012;17:1492–1505.
29. Aaby P, Jensen H, Garly ML, et al. Routine vaccinations and child survival in a war situation with high mortality: effect of gender. Vaccine. 2002;21:15–20.
30. Hirve S, Bavdekar A, Juvekar S, et al. Non-specific and sex-differential effects of vaccinations on child survival in rural western India. Vaccine. 2012;30:7300–7308.
31. Jensen H, Benn CS, Aaby P. DTP in low income countries: improved child survival or survival bias
? BMJ. 2005;330:845–846.
32. Aaby P, Jensen H. Routine vaccination and child survival in Guinea-Bissau. Author's reply to commentary. BMJ. 2001;322:360.
33. Aaby P, Jensen H. Commentary: contrary findings from Guinea-Bissau and Papua New Guinea. Int J Epidemiol. 2005;34:149–151.
34. Aaby P, Benn CS, Nielsen J, et al. DTP vaccination and child survival in observational studies with incomplete vaccination data. Trop Med Int Health. 2007;12:15–24.
35. Fine PE, Williams TN, Aaby P, et al. Epidemiological studies of the ‘non-specific effects’ of vaccines: I – data collection in observational studies. Trop Med Int Health. 2009;14:969–976.
36. Aaby P, Martins CL, Garly ML, et al. Non-specific effects of standard measles vaccine at 4.5 and 9 months of age on childhood mortality: randomised controlled trial. BMJ. 2010;341:c6495.
37. Locht C, Mielcarek N. Live attenuated vaccines against pertussis. Expert Rev Vaccines. 2014;13:1147–1158.
38. Fisker AB, Hornshøj L, Rodrigues A, et al. Effects of the introduction of new vaccines in Guinea-Bissau on vaccine coverage, vaccine timeliness, and child survival: an observational study. Lancet Glob Health. 2014;2:e478–e487.
39. Immunization and Vaccine related Implementation Research Advisory Committee (IVIR-AC): summary of conclusions and recommendations 17-19 September 2014 meeting. Weekly Epidemiological Rec 2015;90:1–8.
APPENDIX: BIAS ASSESSMENT AND SELECTION OF STUDIES
The SAGE review put a major emphasis on assessing bias in the studies included in the review. The assessment of bias was informed by thinking about a hypothetical “target trial” assessing the same comparison as the observational studies. Bias was assessed within 7 domains: (1) bias because of confounding (including frailty bias); (2) bias in participation into the study (including inception bias)—were all eligible children included and did follow-up start at the time of intervention?; (3) bias in measurement of intervention (including survival bias); (4) bias because of departure from intended interventions (performance bias)—were critical cointerventions balanced over intervention groups? If all children received a coadministered vaccine (other than oral polio vaccine (OPV) with DTP) the study was excluded; (5) bias in measurement of outcome (detection bias)—with all-cause mortality as the main outcome there was no problem in this domain; (6) bias because of missing outcome data (attrition bias)—not considered to be problems in this domain; and (7) bias in selection of the reported results (reporting bias)—the reviewers had problems in assessing this but assumed that all studies had “moderate risk of bias.”6 This is a useful overview of bias domains for observational studies.
The Global Advisory Committee on Vaccine Safety (GACVS) and the SAGE reviews differed with regard to several important aspects: First, as a result of point 4 above, the SAGE review excluded studies in which all children received DTP and BCG simultaneously.10 In the GACVS review, most studies had in fact administered BCG and DTP simultaneously.16,17 Second, the SAGE review favored the shortest follow-up period reported in the paper as this would be likely to represent the most direct effect of being vaccinated versus unvaccinated. This is also in contrast to the GACVS review in which follow-up was to 2 years of age even though that would mix the effects of several vaccines.10,12 This latter approach assumes that vaccines have constant effects over time. Specific effects of specific vaccines may well be relative constant through childhood; however, NSEs may be related to reprogramming of innate immune responses; for example, BCG has been shown to reprogram monocytes via epigenetic changes promoting stronger proinflammatory responses.5 Hence, subsequent vaccinations may change the programming. Potential important nonspecific effects will therefore be most visible while a given vaccine is the most recent vaccination and the assessment should therefore not mix the effect of several vaccines over an extended period of follow-up.
Keywords:Copyright © 2016 Wolters Kluwer Health, Inc. All rights reserved.
diphtheria-tetanus-pertussis vaccine; diphtheria-tetanus-pertussis; frailty bias; nonspecific effects of vaccine; survival bias