Secondary Logo

Journal Logo


Methodologic approaches in studies using real-world data to measure pediatric safety and effectiveness of vaccines administered to pregnant women: a scoping review protocol

Lasky, Tamar1; McMahon, Ann W.2; Hua, Wei3; Forshee, Richard1

Author Information
doi: 10.11124/JBISRIR-D-19-00266
  • Free



Maternal vaccination aims to prevent illness in pregnant women and/or confer immunity to the fetus.1-6 In the United States, two vaccines are recommended for all pregnant women to prevent illness in the infant: inactivated influenza vaccine (recommended since 2004) and the combined tetanus-diphtheria-acellular pertussis (Tdap) vaccine (recommended since 2013).7-9 When the mother is vaccinated during pregnancy, both these vaccines may confer immunity and/or protection to the fetus, particularly between birth and when the first infant immunizations are given, when the risk of illness is highest for the infant.1,3 Infants under six months of age can experience severe complications or death if they contract influenza, and influenza vaccine is not approved for infants younger than six months of age.10 Similarly, infants have a high probability of hospitalization and severe complications if they contract pertussis, and they cannot be vaccinated until they are two months of age.3,11 While other vaccines may be administered to pregnant women for maternal protection, no other vaccines are currently recommended to confer immunity to the infant and are thus outside the focus of this review.9

The increased interest in the potential use of real-world data (RWD) to provide real-world evidence (RWE) to support regulatory decision-making around products including vaccines is stated in the U.S. Food and Drug Administration (FDA) framework: “RWD are data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources.”12(p.4) Additionally, “RWE is the clinical evidence about the usage and potential benefits or risks of a medical product derived from analysis of RWD.”12(p.4) Randomized controlled trials have been conducted to estimate efficacy of maternal immunization in preventing influenza and pertussis in infants, as have studies using RWD. While studies show benefits of maternal immunization in preventing infant illness and no safety concerns, there is continued interest in assessing safety, describing the effectiveness of each season's influenza vaccines, and monitoring the population effects on infant pertussis. In addition, it is of interest to policy makers and regulators to describe effects on secondary outcomes such as birth weight or hospitalization, and to describe the duration of benefit. In countries such as the United States, where pertussis and influenza vaccines are recommended for use in pregnant women, ethical issues around randomization in the absence of equipoise may constrain the use of randomized trials, thus increasing the desirability of alternative approaches such as RWE. This is consistent with an increased focus on regulatory use of RWE, and may offer advantages with regards to sample size, population heterogeneity, and costs.

Considerable efforts have been made to develop assessment tools to guide the use of RWD/RWE in a range of policy contexts. One example is the Good Research for Comparative Effectiveness (GRACE) guidelines developed for non-interventional studies of comparative effectiveness to determine which studies are sufficiently rigorous to be reliable enough for use in health technology assessments.13 A second example is the Joint ISPOR/ISPE Special Task Force publication on good practices for RWD studies of treatment and/or comparative effectiveness.14 More specifically, there have been efforts to set guidelines for database selection, and checklists to ensure regulatory-grade data quality have also been published.15,16 None of these efforts have been directed at the highly specialized and unique challenges of studying maternal vaccines administered to prevent disease in the infant.

This scoping review mapping studies using RWD/RWE to measure pediatric safety and effectiveness of vaccines administered to the mother during pregnancy will help clarify approaches to key methodologic issues such as documentation of the vaccine administered, linkage of maternal and infant data, estimation of gestational age, and definition of safety and effectiveness end points. It will also describe the sources of patient information and the electronic health care, administrative claims, and patient registry databases used as the source of clinical data, as well as provide context for planning regulatory approaches to use of RWD and RWE in this area.

The 21st Century Cures Act requires the FDA to develop a program to evaluate the use of RWE to support approval of new indications for approved drugs or to satisfy post-approval study requirements.17 As noted in the FDA Framework released in 2018, “The strength of RWE submitted in support of a regulatory decision depends on the clinical study methodology and the reliability (data accrual and data quality control (data assurance)) and relevance of the underlying data.”12(p.14) The framework also discusses the FDA approach to assessment of RWD and notes, “FDA intends to adapt this approach to assess sources of RWD used to generate RWE of drug product effectiveness, recognizing that the specific elements to consider will likely differ by RWD type and the type of research for which the data are intended.”12(p.15) A librarian developed search strategies to optimize sensitivity and specificity of the electronic search, and the authors will hand search systematic reviews to find studies missed by the electronic search. A preliminary search for existing scoping reviews on the topic has been conducted using the search terms “vaccine,” “vaccinations,” and “scoping review.” The search was conducted in PubMed, Embase, and the JBI Database of Systematic Reviews and Implementation Reports. One current scoping review about vaccines or vaccinations in pregnant women was identified; however, it focused on efforts to improve immunization rates rather than methods of studying effects.18 Thus, this current scoping review will provide regulatory and other decision-makers with an evidence base to inform the evaluation of studies conducted on mothers and infants and using RWD.

Review questions

  • i) What studies using RWD have been conducted to measure pediatric safety and effectiveness of influenza or pertussis vaccines administered to the mother during pregnancy?
  • ii) What electronic health care, administrative claims, or patient registry databases have been used in the above studies?
  • iii) What methods were used to link maternal and infant records?
  • iv) How did the researchers document the vaccine administered?
  • v) What methods were used to estimate gestational age at the time of maternal vaccination?

Inclusion criteria


Studies that include pregnant women and their infants will be included (human studies only). Participants can be from any population, reproductive age, and with any health status.


The concept of interest is the use of RWD to produce RWE to estimate the effect of immunization during pregnancy on the infant. The review will consider any studies of women immunized during pregnancy. Studies of infant immunizations will be excluded. Studies reporting on effectiveness or safety outcomes in the infant will be included.

Effectiveness end points would describe the occurrence of vaccine-preventable disease (influenza or pertussis) in infants born to immunized mothers, and are open-ended in order to capture maximum data on how effectiveness end points have been defined and measured.

Safety end points would describe adverse events in infants born to immunized mothers. This may include still births or spontaneous abortions, pre-term delivery, birth weight, congenital malformations, and other end points. This concept is open-ended in order to capture the full range of safety end points considered in the literature.


Studies conducted in any country will be included. There are no restrictions on cultural factors, geographic location, or specific settings.

Types of sources

Studies using RWD (as defined by the FDA framework) will be included. This may include studies using electronic health records, administrative claims, pharmacy benefit records, or registries. Observational studies using electronic databases of patient health data from electronic health records, administrative claims, pharmacy benefit records, or registries will be included. Letters, reviews (except to search references of systematic reviews), case reports, and animal studies will be excluded.


The proposed scoping review will be conducted in accordance with JBI methodology.19 It follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR).19,20

Search strategy

The search strategy will aim to locate published primary studies. A preliminary search of MEDLINE and Embase was undertaken to identify articles on the topic. The text words contained in the titles and abstracts of identified articles, and the index terms used to describe the articles were analyzed by a reference librarian19 to develop a full search strategy for PubMed and for Embase (Appendix I). The reference lists of systematic reviews identified in the electronic search will be searched to identify articles missed by the electronic search. Scientists working on maternal vaccines at FDA will be queried for additional references missed by the previous searches.

PubMed and Embase will be searched, and colleagues at the FDA will be contacted to identify published studies. Unpublished studies will not be included due to the confidential nature of unpublished studies submitted to FDA.

Articles published during the 10 years prior to the search will be included. The search is limited to the previous 10 years to include the time period in which maternal immunizations have been administered to pregnant women for the purpose of conferring immunity on the infant. Articles published in English will be included.

Study selection

Citations will be managed using EndNote X9 (Clarivate Analytics, PA, USA). Citations from electronic searches will be combined and the duplicates removed. Study selection will take place in two steps: titles and abstracts will be screened for general relevance, and full-text articles will be screened for more detailed inclusion criteria. Two reviewers will conduct each level of screening using a data collection form, and a third expert will be available for adjudication of discrepancies. Initially articles will be included for full-text review if either reviewer recommends inclusion; then agreement between the two reviewers will be required for a study to be included. If the two reviewers disagree on inclusion of the citation, a third reviewer will adjudicate. Studies that do not meet the inclusion criteria will be excluded, and reasons for exclusion will be reported in the review. Results of the search will be presented using a PRISMA-ScR flow diagram.20

Data extraction

Data will be extracted from papers included in the scoping review by two independent reviewers using a data extraction tool developed by the authors. The data extracted will include specific details about the key elements in a spreadsheet format. The items to be extracted are listed in Appendix II. The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included paper. Modifications will be detailed in the full scoping review. Any disagreements that arise between the reviewers will be resolved through discussion, or with a third reviewer. Extraction errors will be minimized by cross-checking results, using a standardized data extraction form, and piloting the data extraction form with reviewers. Data extraction will permit the included studies to be characterized by study features and methodologic features.

Data presentation

The extracted data will be presented in charts or tables in a manner that aligns with the objective of this scoping review. These will be accompanied by a narrative and will reflect the information captured through data extraction. Quantitative analyses will be limited to descriptions of the numbers and percentages of studies addressing a specific issue (for example, number of studies estimating gestational age and the timing of vaccine administration). Narrative summaries and synthesis of findings will follow the JBI methodology for scoping reviews. Quality assessments and risk of bias assessments will not be performed, as the aim of the scoping review is to provide an overview of the map of evidence rather than to answer a particular question.20,21


The scoping review methodology synthesizes information from publicly available publications and is not considered human subject research.


Lee Bernstein, lead librarian, U.S. Food and Drug Administration, Office of the Commissioner, for developing the search strategies.


The protocol has been prepared as part of a project “Database attributes for assessing pediatric safety and effectiveness of vaccines administered to the mother during pregnancy” funded by FDA's Perinatal Health Center of Excellence (PHCE) Intramural Funding Program. The funders have no input in developing the protocol or conducting the review. Funding supports a post-doctoral fellow (JM) who will assist in screening and data extraction.

Appendix I: Search strategies


#1: “Pertussis Vaccine”[Majr]OR “Influenza Vaccines”[Majr]

#2: “Infant”[Mesh] OR infant[TIAB]

#3: “Pregnancy”[Mesh]

#4: #1 AND #2 AND #3

#5: “Treatment Outcome”[Mesh]

#6: efficacy[TIAB] OR effective[TIAB] or safety OR risk

#7: #5 OR #6

#8: #4 AND #7

#9: “Case Reports” [Publication Type] OR “Editorial” [Publication Type]

#10: #8 NOT #9 = 241

#11: pertussis vaccin OR pertussis immuni OR influenza vaccin OR influenza immuni

#12: pregnancy[TIAB] OR maternal[TIAB] OR pregnant[TIAB]

#13: #11 AND #12 AND #7 = 87

#14: #10 AND #13 = 328

#15: #14, Humans only = 241


English; last 10 years

#1: ‘pertussis vaccine’/exp/mj OR ‘influenza vaccine’/exp/mj

#2: ‘pregnancy’ OR maternal OR pregnant

#3: ‘infant’

#4: ‘treatment outcome’ OR effectiv OR efficacy OR safety OR risk

#5: #1 AND #2 AND #3 and #4

#6: ‘case report’/exp OR ‘editorial’/exp ’/exp

#7: #5 NOT #6 = 184

Appendix II: Data charting/extraction fields

Key information that will be charted/extracted:

  • a. Author(s)
  • b. Year of publication
  • c. Country(ies) where the study was conducted
  • d. Electronic health care, administrative claims, or patient registry databases used as source of study data
  • e. Type of electronic health care, administrative claims, or patient registry databases used as source of study data
  • f. Number of electronic health care databases used as source of study data
  • g. Number of pregnant women in study (if not reported, number of infants in the study)
  • h. Vaccines studied (influenza, pertussis, or both)
  • i. Coding system for describing vaccine administered (for example, CVX, ICD-9, ICD-10, HCPCS, other, none)
  • j. Measurement of gestational age at immunization (y/n)
  • k. Safety outcomes
  • l. Effectiveness outcomes
  • m. Method of mother-infant data linkage (e.g. use of a unique identifier, algorithm)
  • n. Method of estimating gestational age (e.g. use of supporting clinical measures, algorithm)


1. Omer SB. Maternal immunization. N Engl J Med 2017; 376 (13):1256–1267.
2. Abu Raya B, Edwards KM, Scheifele DW, Halperin SA. Pertussis and influenza immunisation during pregnancy: a landscape review. Lancet Infect Dis 2017; 17 (7):e209–e222.
3. Bergin N, Murtagh J, Philip RK. Maternal vaccination as an essential component of life-course immunization and its contribution to preventive neonatology. Int J Environ Res Public Health 2018; 15 (5):847–858.
4. Nunes MC, Cutland CL, Madhi SA. Influenza vaccination during pregnancy and protection against pertussis. N Engl J Med 2018; 378 (13):1257–1258.
5. Rasmussen SA, Watson AK, Kennedy ED, Broder KR, Jamieson DJ. Vaccines and pregnancy: past, present, and future. Semin Fetal Neonatal Med 2014; 19 (3):161–169.
6. Vojtek I, Dieussaert I, Doherty TM, Franck V, Hanssens L, Miller J, et al. Maternal immunization: where are we now and how to move forward? Ann Med 2018; 50 (3):193–208.
7. Akinsanya-Beysolow I, Jenkins R, Meissner HC. Advisory Committee on Immunization Practices (ACIP) recommended immunization schedule for persons aged 0 through 18 years--United States, 2013. MMWR supplements 2013; 62 (1):2–8.
8. Harper SA, Fukuda K, Uyeki TM, Cox NJ, Bridges CB. Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 2004; 53 (RR–6):1–40.
9. US Department of Health and Human Services Centers for Disease Control and Prevention. Recommended adult immunization schedule for ages 19 years and older [Internet]. [cited 2020 Mar 29]. Available from:
10. Moriarty LF. Infants and the seasonal influenza vaccine: a global perspective on safety, effectiveness, and alternate forms of protection. Hum Vaccin Immunother 2014; 10 (9):2721–2728.
11. Furuta M, Sin J, Ng ESW, Wang K. Efficacy and safety of pertussis vaccination for pregnant women – a systematic review of randomised controlled trials and observational studies. BMC Pregnancy Childbirth 2017; 17 (1):390.
12. US Food and Drug Administration. Framework for FDA's real-world evidence program [Internet]. 2018 [cited 2020 Mar 29]. Available from:
13. Dreyer NA, Bryant A, Velentgas P. The GRACE checklist: a validated assessment tool for high quality observational studies of comparative effectiveness. J Manag Care Spec Pharm 2016; 22 (10):1107–1113.
14. Berger ML, Sox H, Willke RJ, et al. Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE Special Task Force on real-world evidence in health care decision making. Pharmacoepidemiol Drug Saf 2017; 26 (9):1033–1039.
15. Hall GC, Sauer B, Bourke A, Brown JS, Reynolds MW, LoCasale R. Guidelines for good database selection and use in pharmacoepidemiology research. Pharmacoepidemiol Drug Saf 2012; 21 (1):1–10.
16. Miksad RA, Abernethy AP. Harnessing the power of real-world evidence (RWE): a checklist to ensure regulatory-grade data quality. Clin Pharmacol Ther 2018; 103 (2):202–205.
17. 114th US Congress. 21st Century Cures Act. HR 34. In: 114th US Congress, ed. Public Law 114-255. Washington, DC2016.
18. MacDougall DM, Halperin SA. Improving rates of maternal immunization: challenges and opportunities. Hum Vaccin Immunother 2016; 12 (4):857–865.
19. Peters M, Godfrey CM, McInerney P, Baldini Soares C, Khalil H, Parker D. Aromataris E, Munn Z. Chapter 11: scoping reviews. JBI Reviewer's Manual [Internet]. Adelaide: JBI, 2017 [cited 2020 Mar 29]. Available from:
20. Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 2018; 169 (7):467–473.
21. Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 2018; 18 (1):143.

immunization; infant; influenza; pertussis; pregnancy

© 2020 JBI