In recent years, the evaluation of nucleic acid amplification tests (NAATs) for detecting Chlamydia trachomatis and Neisseria gonorrhea is based on a methodology called the patient-infected-status algorithm (PISA). In the simplest version of PISA, 4 test-specimen combinations (comparator tests) are used to define the gold standard. If a person shows a positive result by any 2 or more of these 4 comparator tests, the person is classified as infected; otherwise, the person is considered to be uninfected. A new test is then compared with this diagnostic algorithm. PISA-based sensitivity and specificity estimates of nucleic acid amplification tests have been published in the medical and microbiologic literature and have been included in FDA-approved package inserts of NAATs for detecting C. trachomatis. Using simulations, we compare 2 versions of the patient-infected-status algorithm with latent-class models and an imperfect gold standard. We show that the PISA can produce highly biased test-performance parameter estimates. In a series of simulated scenarios, none of the 95% confidence intervals for PISA-based estimates of sensitivity and prevalence contained the true values. In addition, the PISA-based estimates of sensitivity and specificity change markedly as the true prevalence changes. We recommend that PISA should not be used for estimating the sensitivity and specificity of tests.
From the aDivision of STD Prevention, National Centers for Disease Control and Prevention, Atlanta, GA; bTechnology Assessment Unit, McGill University Health Center, Montreal, Quebec, Canada; cDepartment of Epidemiology and Biostatistics, McGill University, Montreal, Quebec, Canada; and dDepartment of Statistics, University of British Columbia, Vancouver, British Columbia, Canada.
Submitted 16 April 2011; accepted: 6 September 2011.
The views expressed in this article are those of the authors and do not necessarily reflect the views or policies of the CDC or the US Public Health Service.
Supported by a Chercheur Boursier Junior 2 award from the Fonds de la Recherche en Sante' du Quebec. The authors reported no other financial interests related to this research.
Correspondence: Alula Hadgu, Division of STD Prevention, Centers for Disease Control, 1600 Clifton Road, Atlanta, GA 30333. E-mail: AHadgu@cdc.gov.