Qualitative Assessment of Pertussis Diagnostics in United States Laboratories

Abstract

Background: 

United States national surveillance data show that the use of culture for pertussis diagnostics has sharply declined, whereas polymerase chain reaction (PCR) is now the most common testing method. PCR testing for pertussis is rapid and sensitive, but the lack of standardization and variable specificity is concerning.

Methods: 

A web-based survey containing 12 questions was sent to public health, commercial and hospital-based US laboratories performing clinical diagnostics to determine the pertussis diagnostics used. An extensive real-time PCR (RT-PCR) questionnaire accompanied a proficiency panel assessing the types of extraction methods, RT-PCR methods and current quality control in place at the laboratories. The proficiency panel of 12 specimens containing Bordetella pertussis at various concentrations and negative controls was created to detect cross-contamination and assess the lower limit of detection.

Results: 

One hundred twenty-three (35%) of 355 respondents from the web-based survey performed diagnostic tests for the presence of B. pertussis. Eighty-three (71%) labs reported performing culture, whereas 67 (54%) labs used PCR. All 41 laboratories that consented to participate in the proficiency exercise used the IS481 RT-PCR target; however, a variety of extraction and RT-PCR methods were employed. The laboratories correctly identified 92% of the B. pertussis specimens, and 5% of the laboratories (1.8% of the panel specimens) reported at least 1 false-positive.

Conclusions: 

The small percentage of false-positives suggests that adequate procedures are in place to prevent cross-contamination. Differing extraction and PCR methods as well as variable analytic sensitivity emphasize the necessity for an external well-defined quality control program and interlaboratory pertussis PCR harmonization.

Pertussis is a highly contagious respiratory syndrome caused by Bordetella pertussis. Pertussis-like illness in humans may be caused by 2 additional Bordetella spp., Bordetella holmesii^1–3 and Bordetella parapertussis,⁴ whereas Bordetella bronchiseptica is an infrequent human pathogen. Since the inclusion of polymerase chain reaction (PCR) in the Council of State and Territorial Epidemiologist surveillance case definition for pertussis in 1997, the proportion of reported cases diagnosed using culture isolation has sharply declined and PCR is now the predominant testing method.⁵ PCR offers improved sensitivity over culture, but the majority of tests used in the United States for pertussis diagnosis include the IS481 target, which cannot discriminate between B. pertussis and B. holmesii, and its high copy number makes it susceptible to cross-contamination.^6–8

European countries^9–12 have implemented proficiency programs to evaluate their real-time PCR (RT-PCR) diagnosis of pertussis. To undertake a similar effort in the United States, we administered a web-based survey to diagnostic laboratories and distributed proficiency panels to the respective participants. The initial web-based survey was designed to determine which laboratories conducted pertussis testing, what tests (culture, PCR, serology or direct fluorescent antibody staining) were used to diagnose pertussis, what RT-PCR targets were used to detect B. pertussis and whether the laboratories would consent to participate in the proficiency study. All laboratories included in the proficiency study completed a comprehensive RT-PCR questionnaire to assess the specific nucleic acid extraction and RT-PCR methods to ascertain if any commonality existed among laboratories. The proficiency panel assessed the ability to identify B. pertussis, the limit of detection and the false-positivity rate.

METHODS

Web-based Survey

A web-based survey was developed, which included 12 questions organized in 5 different sections including: the type of test(s) used to diagnose pertussis, the number of specimens analyzed by either conventional or real-time PCR in a 6-month period, basic PCR information (if done), utilization of serology in pertussis diagnostics and whether the laboratory would agree to participate in a RT-PCR proficiency panel. Directories of laboratory contacts were compiled for every state of laboratories with significant testing capacity including hospital laboratories, commercial diagnostic companies and public health laboratories. Because no centralized list of testing laboratories exists for the entire United States, various sources, including our state public health partners, internet searches and public directories were used to generate this list. Lack of participation in the proficiency testing did not preclude a laboratory from completing the survey.

Questionnaire

A more detailed questionnaire inquiring about the specific extraction and RT-PCR methods was sent with the proficiency panel. The questionnaire contained 17 queries encompassing type of extraction method, positive and negative controls for extraction and RT-PCR, the type of platform and RT-PCR setup as indicated in Table, Supplemental Digital Content 1, https://links.lww.com/INF/B556. After results of survey and panels were obtained, brief telephone interviews were conducted with each of the participating laboratories to clarify responses to ensure accuracy of results.

Proficiency Panel

The proficiency panel contained 8 blinded saline-based heat-inactivated B. pertussis samples and 4 negative samples prepared at a Bordetella-free (naïve) facility in Newark, NJ. Physiological saline solution (0.85%) was inoculated with cells from a single B. pertussis (A639) colony grown on Regan-Lowe charcoal agar. B. pertussis strain A639 was chosen as it is representative of the circulating strains in the United States and is the reference standard for pulsed field gel electrophoresis typing.¹³ Ten-fold serial dilutions of B. pertussis A639 were prepared in saline, and the number of colony forming units (cfu) in each dilution was determined. The panel was pretested at Centers for Disease Control and Prevention (CDC) using the MagNAPure LC-DNA Isolation Kit III (Roche, Indianapolis, IN) and the multitarget RT-PCR algorithm for Bordetella spp., which includes the multiplex (IS481, pIS1001 and hIS1001) and the ptxS1 assays on the ABI7500 (Applied Biosystems Inc., Carlsbad, CA).¹⁴ The order of the 12 samples in the panel is indicated in Table 1. The panel with 500 µL of each heat-inactivated specimen was shipped on ice packs to the participating laboratories. Laboratories were asked to perform their standard procedures for nucleic acid extraction and RT-PCR testing. Descriptive analyses were done using arithmetic means and associated coefficients of variation.

TABLE 1: Centers for Disease Control and Prevention RT-PCR Results of Proficiency Panel With Multitarget Bordetella spp. Assay

RESULTS

Web-based Survey

Invitations were e-mailed to 747 laboratories with 29 e-mails returned as undeliverable. Mailed invitations were sent to 2467 laboratories requesting participation; however, the number not received is unknown. A total of 355 laboratories completed the survey from all the invitations sent. Of these 355 laboratories, 123 (35%) conduct pertussis testing including 18 (15%) public health, 22 (18%) commercial and 83 (67%) hospital-based laboratories. Sixty percent of laboratories reported testing between 101 and 1000 specimens by PCR in a 6-month period and 7% reported testing more than 1000.

Serology

Serology was performed by 8 (7%) laboratories (5 of which were commercial laboratories). Along with serology, 3 of these 8 laboratories offered both culture and PCR, 3 offered culture and 2 offered PCR. Serologic testing kits varied between laboratories, as different antigens were utilized and multiple immunoglobulins were detected.

Culture

Culture was performed by 83 (71%) laboratories which were 71%, 50% and 94% of the hospital-based, commercial and public health laboratories, respectively. Direct fluorescent antibody staining was performed by 17 (15%) of laboratories.

PCR

Sixty-seven (54%) laboratories performed PCR, of those only 29 (43%) also performed culture and 55% were public health laboratories. Ninety-three percent used RT-PCR methods. Eighty-one percent used the IS481 target alone or in combination with other targets for B. pertussis, whereas IS1001 was used for B. parapertussis (31%). RT-PCR methods varied substantially with multiple extraction methods, laboratory developed tests, commercially available assays and platforms.

Response to Proficiency Testing

Forty-one laboratories, 33.3% of those performing pertussis testing, consented to receive the proficiency panel.

PCR Questionnaire

A variety of manual (54%) and automated (46%) nucleic acid extraction methods were used by 39 of 41 participating laboratories (see Table, Supplemental Digital Content 1, https://links.lww.com/INF/B556) and 2 laboratories did not perform nucleic acid extractions. All laboratories targeted the IS481 region; however, other additional targets were included such as IS1001 (23 laboratories), ptxS1 (3), recA (2) and prn (1). More commercial Analyte Specific Reagents from Roche, Cepheid (Sunnyvale, CA) or Eragen Biosciences (Madison, WI) were used (56%) than laboratory developed tests (44%) by the 41 participants. The majority used LightCycler (Roche) or Smartcycler (Cepheid) instruments with 45 or fewer cycles per RT-PCR reaction (80%). Separate areas/rooms and appropriate caution for addition and manipulation of positive controls were reported by most laboratories, which is necessary for accurate PCR and recommended by CDC. However, significant variability existed in the cycle threshold (Ct) cutoff values (range: 30–50 cycles) as well as whether repeat testing was performed including laboratories using the same commercial Analyte Specific Reagents. These results demonstrated the extensive variability that exists in both the type of extraction methods and PCR assays utilized by all types of laboratories.

PCR Proficiency Panels

The CDC real-time PCR assay served as the reference standard for the proficiency panel (Table 1). Forty-one laboratories tested the proficiency panel and 92% of the specimens were correctly identified as B. pertussis. Five percent of the laboratories (1.8% of the specimens) reported at least 1 false-positive. IS481 Ct values for B. pertussis samples had coefficients of variation ranging from 9.2% to 14.1% (Table 2). The percentage of laboratories reporting false-negatives was correlated with the number of colony forming units of B. pertussis in each sample. The 2 samples at 1000 cfu were correctly identified by 95% and 100% of the laboratories, whereas only 76% and 88% could identify the 2 B. pertussis samples at 1 cfu. These samples encompass the range of concentrations in clinical specimens normally received for PCR testing.

TABLE 2: Proficiency Panel Results for Bordetella pertussis Samples

The assays used by the laboratories were not the CDC multitarget RT-PCR assay. The Ct value for the proficiency panel specimens with the CDC assay (Table 1) was lower than the mean Ct value from the participating laboratories (Table 2). However, the range of values from the participating laboratories demonstrated that some laboratories obtained lower Ct values with their assay than the results with the CDC assay. The laboratories that reported a false-positive or false-negative used either manual or automated extraction methods, and thus, a concordance with the type of extraction method could not explain inaccurate results.

DISCUSSION

The web-based survey response of 355 laboratories is higher than expected because the survey was sent to all identified diagnostic laboratories without knowledge of whether they conducted pertussis testing. Only a few laboratories reported using serology perhaps due to the lack of harmonization that exists with serologic commercial kits.^12,15,16 A limited number of laboratories performed direct fluorescent antibody staining, as it is not recommended as a diagnostic assay due to its lack of sensitivity and specificity. In the survey, we found more laboratories performing culture than PCR, which differs from the testing trends observed in the US national surveillance data. This difference is not surprising because a relatively small number of large commercial laboratories conduct the majority of pertussis testing in the United States. Of the laboratories completing our survey, only 18% of the survey respondents were from commercial testing laboratories. The excellent response of 41 laboratories agreeing to test the proficiency panel was an indicator of the unmet need for proficiency testing and was significantly larger than the number of laboratories assessed in proficiency exercises performed in other countries.^9–12

Sensitivity of the RT-PCR assays used by participating laboratories was adequate allowing identification of B. pertussis in over 90% of the specimens from the panel. The small percentage of falsely positive results from the panel suggest adequate procedures are in place to prevent within laboratory cross-contamination.⁶ In this study, the incorrect results did not correlate with the utilization of manual or automated extraction methods; however, automated extraction methods are recommended to extract nucleic acids. With manual extraction methods, an increased risk of contamination exists and with multicopy targets this risk is amplified.

The use of single target assay, IS481, does not allow differentiation between B. pertussis and B. holmesii or the ability to detect B. parapertussis. Recent publications suggest that these pathogens may markedly contribute to the total burden of pertussis-like illness.^1–3 For these reasons, recently we prepared proficiency panels to test the ability of laboratories to discriminate between Bordetella spp. The potential pitfalls of IS481 due to the large number of copies in the B. pertussis genome has been described in recent pseudo outbreaks^7,8 and is an inherent problem with such a sensitive target. Webinars and best practices guidance have been developed by CDC to help healthcare professionals optimize the use of PCR testing for pertussis by avoiding some of the more common pitfalls leading to inaccurate results (http://www.cdc.gov/pertussis/clinical/diagnostic-testing/diagnosis-pcr-bestpractices.html).

To overcome the problems of a single target assay, a multitarget RT-PCR assay was developed at CDC, which includes a multiplex assay with multicopy targets for B. pertussis (IS481), B. holmesii (hIS1001) and B. parapertussis (pIS1001).¹⁴ In addition to the multiplex assay of insertion sequences, primers and probe that target the coding region of subunit 1 of the pertussis toxin gene (ptxA gene) are maintained in our diagnostic algorithm to enhance species determination, to increase the accuracy of B. pertussis DNA detection by RT-PCR¹⁷ and to determine coinfections. Diagnostic RT-PCR assays using only virulence genes to determine the presence of B. pertussis DNA have been described.^18,19 However, recently in France, Japan, Finland and United States, it was found that B. pertussis isolates did not express pertussis toxin and/or pertactin because the genes for these proteins were deleted or modified.^20–23 Clinical specimens and/or isolates such as these would confound diagnostics based solely on these virulence genes. In outbreak settings, RT-PCR results must be correlated with the results of other laboratory tests, such as culture and serology,²⁴ or with clinical and epidemiological data before the etiological agent of the outbreak is confirmed as B. pertussis.

The different nucleic acid extraction methods as well as the RT-PCR interpretation criteria and cutoff values used by the laboratories highlights the lack of standardization between laboratories. The variable analytic sensitivity and the lack of additional targets to identify other Bordetella spp. emphasize the need for validated and harmonized pertussis RT-PCR assays to improve pertussis diagnosis. Our surveys and proficiency panel demonstrate the importance and crucial role of RT-PCR in pertussis diagnostics and moreover exemplify the need for an external well-defined quality control program to monitor the performance of RT-PCR assays.

ACKNOWLEDGMENTS

We thank Dr. Peter Trabold, Dr. Gregory Chiklis and personnel at Zeptometrix for advice, preparation and distribution of the panels. We would especially like to thank the laboratories participating in the evaluation of the pertussis laboratory panels: Baptist Medical Center, Jacksonville, FL; Children’s Healthcare of Atlanta, Atlanta, GA; Children’s Hospital and Regional Medical Center, Seattle, WA; Children’s Hospital Central California, Madera, CA; Children’s Hospital of Philadelphia, Philadelphia, PA; Children’s Medical Center, Dallas, TX; Cook Children’s Medical Center, Fort Worth, TX; Duke University Health System, Durham, NC; Florida DOH Bureau of Laboratories, Jacksonville, FL; Focus Diagnostics, Cypress, CA; Health Network Laboratories, Allentown, PA; Idaho Bureau of Laboratories, Boise, Idaho; Illinois Department of Public Health, Chicago, IL; The Johns Hopkins Hospital, Baltimore, MD; Kaleida Health/Women & Children’s Hospital, Buffalo, NY; Laboratory Alliance of Central New York, Liverpool, NY; Labs NW/MultiCare Health System, Tacoma, WA; Legacy Central Laboratory, Portland, OR; Louisiana Department of Health, Metairie, LO; Madigan Army Medical Center, Tacoma, WA; Marshfield Laboratories, Marshfield, WI; Mercy Integrated Laboratories, Toledo, OH; Methodist Hospital, San Antonio, TX; Michigan Department of Community Health, Lansing, MI; New Hampshire Public Health Laboratory, Concord, NH; North Carolina State Laboratory of Public Health, Raleigh, NC; Northwest Community Hospital, Arlington Heights, IL; Norton Healthcare, Louisville, KY; Oregon State Public Health Laboratory, Hillsboro, OR; Quest Diagnostics, Denver, CO; Rady Children’s Hospital, San Diego, CA; Seattle-King County Public Health Laboratory, Seattle, WA; South Carolina DHEC Bureau of Laboratories, Columbia, SC; Southwest Washington Medical Center, Vancouver, WA; Sparrow Regional Laboratories, Lansing, MI; St. Louis Children’s Hospital, St. Louis, MO; Stanford University, Palo Alto, CA; Texas Children’s Hospital, Houston, TX; University of Michigan Health System, Ann Arbor, MI; The University of North Carolina Hospitals, Chapel Hill, NC; Waukesha Memorial Hospital, Waukesha, WI; West Virginia Office of Laboratory Services, South Charleston, WV; Wisconsin State Laboratory of Hygiene, Madison, WI.