Secondary Logo

Journal Logo

Institutional members access full text with Ovid®

The use of test-negative controls to monitor vaccine effectiveness

a systematic review of methodology

Chua, Huiying1; Feng, Shuo1; Lewnard, Joseph A.2; Sullivan, Sheena G.3,4,5; Blyth, Christopher C.6,7,8; Lipsitch, Marc9,10; Cowling, Benjamin J.1

doi: 10.1097/EDE.0000000000001116
Review Article: PDF Only

Background: The test-negative design is an increasingly popular approach for estimating vaccine effectiveness (VE) due to its efficiency. This review aims to examine published test-negative design studies of VE and to explore similarities and differences in methodological choices for different diseases and vaccines.

Methods: We conducted a systematic search on PubMed, Web of Science, and Medline, for studies reporting the effectiveness of any vaccines using a test-negative design. We screened titles and abstracts, and reviewed full texts to identify relevant articles. We created a standardized form for each included article to extract information on the pathogen of interest, vaccine(s) being evaluated, study setting, clinical case definition, choices of cases and controls, and statistical approaches used to estimate VE.

Results: We identified a total of 348 articles, including studies on VE against influenza virus (n=253), rotavirus (n=48), pneumococcus (n=24), and nine other pathogens. Clinical case definitions used to enroll patients were similar by pathogens of interest but the sets of symptoms that defined them varied substantially. Controls could be those testing negative for the pathogen of interest, those testing positive for non-vaccine type of the pathogen of interest, or a subset of those testing positive for alternative pathogens. Most studies controlled for age, calendar time, and comorbidities.

Conclusions: Our review highlights similarities and differences in the application of the test-negative design that deserve further examination. If vaccination reduces disease severity in breakthrough infections, particular care must be taken in interpreting vaccine effectiveness estimates from test-negative design studies.

1. World Health Organization Collaborating Centre for Infectious Disease Epidemiology and Control, School of Public Health, The University of Hong Kong, Hong Kong Special Administrative Region, China.

2. Division of Epidemiology, School of Public Health, University of California, Berkeley, Berkeley, California, USA.

3. WHO Collaborating Center for Reference and Research on Influenza at the Peter Doherty Institute for Infection and Immunity, Melbourne, Victoria, Australia.

4. Department of Epidemiology, Fielding School of Public Health, University of California, Los Angeles, Los Angeles, California, USA.

5. Centre for Epidemiology and Biostatistics, School of Population and Global Health, University of Melbourne, Melbourne, Victoria, Australia.

6. Division of Paediatrics, School of Medicine, Faculty of Health and Medical Sciences, The University of Western Australia, Perth, Western Australia, Australia.

7. Wesfarmers Centre of Vaccines and Infectious Diseases, Telethon Kids Institute, Perth, Western Australia, Australia.

8. Department of Infectious Diseases, Perth Children's Hospital, Perth, Western Australia, Australia.

9. Department of Epidemiology, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA.

10. Center for Communicable Disease Dynamics, Harvard T. H. Chan School of Public Health, Boston, Massachusetts, USA.

ACKNOWLEDGMENTS: We thank Julie Au for administrative support.

POTENTIAL CONFLICTS OF INTEREST: BJC has received honoraria from Sanofi Pasteur and Roche. The authors report no other potential conflicts of interest.

FUNDING: This work was supported by a commissioned grant from the Health and Medical Research Fund of the Food and Health Bureau of Hong Kong, and the Research Grants Council of the Hong Kong Special Administrative Region (project no. T11-705/14N). ML and BJC are supported by the Harvard Center for Communicable Disease Dynamics from the National Institute of General Medical Sciences (grant no. U54 GM088558). The WHO Collaborating Centre for Reference and Research on Influenza is supported by the Australian Government Department of Health. The funding bodies had no role in study design, data collection and analysis, preparation of the manuscript, or the decision to publish.

Corresponding author: Prof. Benjamin J Cowling, School of Public Health, Li Ka Shing Faculty of Medicine, The University of Hong Kong, 7 Sassoon Road, Pokfulam, Hong Kong. Tel: +852 3917 6711; Fax: +852 3520 1945; email:

Copyright © 2019 Wolters Kluwer Health, Inc. All rights reserved.