Pneumonia ranks among the top 3 diagnoses in hospital admissions in sub-Saharan Africa.1 In contrast to the United States,2 where most hospitalizations for pneumonia are among children and the elderly, the typical African patient is a young, economically productive adult.3–5 These 2 observations suggest that a moderately priced public-health-intervention strategy against pneumonia is likely to be justifiable in Africa. For children in the developing world, there is considerable international momentum to prevent pneumonia using conjugate vaccines against Haemophilus influenzae type b and Streptococcus pneumoniae subsidized by the Global Alliance for Vaccines and Immunization.6,7 This raises the prospect that adult vaccination may also become feasible in Africa. However, the effectiveness of conjugate pneumococcal vaccine in adults against clinically defined pneumonia will depend critically on the proportion of hospitalized pneumonia cases that are caused by S. pneumoniae.8
S. pneumoniae is invariably found more frequently than any other pathogen in etiological series of pneumonia in Africa.3,5,9 However, the clinical evidence is scarce and, as in the United States and United Kingdom,10,11 the causal agent remains undiagnosed for a substantial proportion of pneumonia cases. Diagnosis is hampered by the lack of a gold standard. Although blood and lung-aspirate cultures are generally regarded as highly specific,12,13 they typically lack sensitivity, and antimicrobial treatment makes the evaluation of culture-based diagnostic tests yet more complicated. Nonetheless, the common clinical treatments in the developing world, (penicillin or ampicillin) reflect the clinical impression and widespread belief that many of the pneumonia episodes without known cause are pneumococcal.4,14–16
The objectives of this study were to define the proportion of episodes of community-acquired pneumonia that are caused by S. pneumoniae, and describe the influence of antimicrobial treatment on the accuracies of diagnostic tests. The study combined data from 5 validation studies of diagnostic techniques for pneumococcal pneumonia.5,17–20 Because there is no gold standard for pneumococcal diagnosis, we applied latent class analysis, a statistical method for estimating the unobserved latent disease status on the basis of several imperfect test results.
Cases consisted of 281 adults with acute community-acquired pneumonia consecutively admitted to Coast Provincial General Hospital (Mombasa) and Kilifi District Hospital (Kilifi, Kenya) between March 1994 and May 1996.5 Each case met all of the following criteria: an acute illness of up to 14 days duration; at least 2 of the symptoms cough, sputum production, breathlessness, pleuritic chest pain, hemoptysis or fever; evidence of consolidation on posterioanterior or lateral chest radiographs judged by the study physician and later confirmed by a consultant radiologist5; no admission to hospital in the previous 14 days; and informed consent to participate to the study. Samples included blood, sputum, and nasopharyngeal swabs for culture, lung-aspirate material from those in whom an aspirate could be performed safely, urine, and acute sera. Convalescent sera were obtained in survivors who attended a follow-up appointment. Mortality was 10%. The study was approved by the ethical review committees of the Kenya Medical Research Institute and of the London School of Hygiene and Tropical Medicine.
Two aliquots of 10 mL of blood were drawn by separate venipunctures at presentation. Blood and lung-aspirates were cultured in 40 mL of brain-heart infusion broth at 37°C in 5% CO2 and subcultured to 5% horse blood agar and chocolate agar. S. pneumoniae was identified by colonial morphology, sensitivity to optochin, and capsular serotyping by Quellung reaction. Serotyping ambiguities were resolved by the Public Health Laboratory in Oxford, UK.21 Homogenized sputum samples were cultured on blood agar with 5 mcg/mL gentamicin, chocolate agar with 10,000 U/mL bacitracin, and MacConkey agar. Cotton-tipped, flexible, twisted-wire swab specimens were obtained from the posterior nasopharynges of subjects and inoculated onto horse blood agar with 5 mcg of gentamicin/mL. α-Hemolytic colonies were examined for optochin sensitivity and serotyped by the Quellung reaction.
Serologic and Molecular Methods
Full methodologic details are contained in the cited references. In the urine antigen test,17 agglutination of latex particles sensitized with serogroup-specific rabbit antisera indicates the presence of capsular polysaccharide in urine.22 The capsular antigen detection assay was conducted 10 times on each blinded urine sample to cover serogroups 1, 4, 5, 6, 7, 9, 12, 14, 19, and 22. An enzyme immunoassay was used to identify antipneumococcal immune complexes.18 After precipitation of immune complexes from sera using polyethylene glycol in sodium borate, an enzyme immunoassay was performed for antibodies to pneumolysin, C-polysaccharide, and 23-valent polysaccharide vaccine (Pneumovax, Merck). In an ELISA for Anti-PsaA,19 antibodies binding to microtiter plates coated with pneumococcal surface adhesin A (PsaA) were identified with an enzyme-conjugated mouse monoclonal antihuman immunoglobulin G. A ratio rise in antibody concentration between acute and convalescent sera of ≥2-fold was considered positive23 (anti-PsaA rise). Polymerase chain reaction (PCR) for PsaA20 was performed as follows. A single pair of primers, 8229.p and 56496.n (which define an 838-bp fragment of the PsaA gene of S. pneumoniae), were used in a standard PCR on microbial DNA extracted from lung aspirate specimens.24 In addition, to determine the HIV status of the patients, standard methods for HIV antibody positivity were used.5
Bioassay for Antimicrobial Activity in Urine
Urine samples were collected before administration of therapeutic antibiotics (n = 248) and the presence of antibiotics was inferred by the urine's capacity to inhibit the growth of a fully sensitive Staphylococcus aureus (NCTC 6571). Each patient was also questioned about the use of antibiotics during the previous 2 days and during the preceding week.
Latent Class Analysis
Latent class analysis has been extensively applied in assessing accuracies of diagnostic tests when the gold standard is not available.13,25–28 In latent class analysis it is assumed that the observed association between the tests is fully explained by the unobserved (latent) class variable, ie, the true pneumococcal disease status (no/yes).
More formally, consider k tests (k ≥3) where each test Xi, i = 1, …, k can have 2 values: 1 if the test is positive for pneumococcus, and 0 if the test is negative. There are altogether 2k possible combinations of the test results. By utilizing the rule of total probability, these 2k probabilities pr(X1 = x1, X2 = x2, …, Xk = xk), xi = 0, 1, can be expressed in terms of the latent class D as follows:
where pr(D = 1) denotes the proportion of S. pneumoniae among pneumonia patients, pr(Xi = 1 |D = 1) denote the sensitivities of individual tests, and pr(Xi = 0|D = 0) denote their specificities. By assuming a multinomial distribution for the combination probabilities, the parameters can be estimated by optimizing the criterion of maximum likelihood.
The appeal of latent class analysis is that it provides estimates for the unobserved quantities of interest, namely the prevalence of the disease and the specificities and the sensitivities of the tests. However, the assumptions underlying the methodology need to be carefully scrutinized.
The Conditional Independence Assumption
The standard latent class analysis model makes an essential assumption that the tests are independent of each other, conditional on the pneumococcal disease status. When applied to the analysis of diagnostic-test accuracies, this assumption has been justifiably questioned.29 For example, if the obtained sample has been contaminated with pneumococcus and 2 diagnostic tests (eg, culture and PCR) are performed on the same sample, it is more likely that both tests would indicate a false-positive result than if the tests were conditionally independent.
Conditional dependence can be accounted for in latent class analysis. For example, similarities between 2 diagnostic tests can be addressed by the direct-effects approach.28 Alternatively, an additional continuous or categorical latent variable can be imposed on the conditional probabilities.26 However, care is needed in formulating these latent structures, because misspecification of the dependence structure may result in even greater bias.27 A practical alternative approach is to compare the fits of a model assuming conditional independence and dependence between all possible pairs of tests using the likelihood ratio test, and then select a set of diagnostic tests for which the conditional independence assumption holds. Although this means that the accuracies of certain tests are left unanalyzed, this is a particularly useful approach if the parameter of primary interest is the prevalence of pneumococcal disease.
Treatment of Missing Data
A ubiquitous problem in the analysis of multiple diagnostic tests is that some observations are missing from the study subjects. The simplest analytic approach is to exclude all subjects without completely observed test results. However, this reduces the power of the analysis and may introduce selection bias, ie, subjects for whom all the test results are available may not be a representative sample of the population.
When fitting models to all available data, 2 ignorable missing-data mechanisms can be distinguished that have different implications on inference regarding the model parameters30: missing completely at random (MCAR), and missing at random (MAR). If the data are MCAR, it is assumed that those with missing samples originate from the same population as those with complete observations. This assumption may be too strict for the dataset in question. For example, some samples may be difficult to obtain from severely ill patients. Consequently, there may be differences in this subpopulation with regards to pneumococcal disease burden. Under the assumption of MAR, the probability that a missing test is positive can differ for those with partly observed test combinations as long as this probability is equal to the probability for those with complete data, conditional on the other observed test results.
In our latent class analysis, we fitted models to the dataset of all patients, and assumed that the unobserved test results were missing at random (MAR).30 Although there is no explicit way to test the assumption of MAR, it is generally considered a tenable assumption in multivariate analysis with missing data.31 Furthermore, if the data are erroneously assumed to be MAR, this often has only minor impact on the estimates and standard errors.32
Accounting for Antimicrobial Exposure
Antimicrobial exposure reduces the sensitivity of culture-based tests.33 This causes complex difficulties in the analysis of diagnostic accuracy in the absence of a gold standard. If the patients taking antibiotics have a higher proportion of false-negative results for culture-based tests, not only will the culture-based test parameters be biased, but those that are not affected by antibiotics will also be biased because the culture-based tests fail to provide support for the true-positives found by those other tests. A common analytic approach is to exclude those with evidence of antimicrobial treatment. As with missing data, this reduces power, and the subgroup not receiving antibiotics may not be a representative sample of the population.
We adopted 2 approaches to account for antibiotic exposure. In our primary analysis (Model A), we coded culture results in the antibiotic-treated group as missing, and assumed MAR. That is, the probabilities of positive culture results are assumed to be equal in both the antibiotic and nonantibiotic groups, conditional on the results of the non–culture-based tests. Second, we conducted an analysis (Model B) in which all culture results were included as observed, but the specificities and the sensitivities of the culture tests were allowed to vary across the 2 antibiotic groups. This approach was taken because antimicrobial treatment is common, and the accuracies of the culture-based tests during antimicrobial exposure may themselves be of interest. To determine the magnitude of bias, we also fitted a model that took no account of antimicrobial exposure (Model C).
Exploratory Analysis Results
In the preliminary analyses, none of the immunocomplex enzyme immunoassays were associated with blood and lung-aspirate cultures, which are generally regarded as highly specific. This accords with earlier findings that these immunoassays provide little or no predictive value in the pneumococcal pneumonia diagnosis.34 The leftmost column of Table 1 reports the summary of the available tests performed for 281 pneumonia patients after excluding these immunoassay results.
Results on Antimicrobial Exposure
We determined the antimicrobial exposure of patients on the basis of the urinary antimicrobial assay. Based on this gold standard, the sensitivity of self-reported antibiotic use in the last 2 days was 0.51 (69/135) and specificity was 0.78 (88/113). This discrepancy between self-reports and urine tests is probably explained by the use of antimalarials, which are a common first choice of treatment for fever, and some of which have antibiotic properties. Of patients who reported taking no antibiotics in the last 2 days, 57% had evidence of antimicrobial activity in urine. All of these patients reported taking no antibiotics for the whole of the preceding week. The rightmost columns of Table 1 report the percentages of positive results of 7 pneumococcal tests in groups with and without urinary evidence of antimicrobial treatment. Of urine samples from 281 patients, 135 were positive for antimicrobial activity and 113 were negative; urine was not obtained from 33 patients before inpatient antibiotic therapy started. The results are consistent with previous findings that antimicrobial treatment eradicates positive bacterial results when using culture-based tests.33
Latent Class Analysis Results
The dataset for the primary analysis and model diagnosis contained all 7 tests listed in Table 1. However, out of 281 patients, only the culture results of those not under antimicrobial treatment were used (113 patients). The model fit to the data was significantly better when assuming conditional dependence between lung-aspirate culture and PCR (likelihood ratio test χ2 = 7.8; P = 0.02 with 2 df), and between sputum culture and nasopharyngeal swab culture (likelihood ratio test χ2 = 8.4; P = 0.02 with 2 df). Both are plausible findings, considering the shared location from which the specimens were obtained. Since lung-aspirate culture is regarded as highly specific, and nasopharyngeal swab culture prevalence was considerably higher than that of sputum culture, we omitted lung-aspirate PCR and sputum culture, leaving 5 tests in the final analysis. Consequently, since culture results of patients under antimicrobial treatment were coded missing for this analysis, 5 patients were missing all test results. Thus, the final number of patients analyzed was 276.
When no parameter constraints were applied, the specificities of both blood and lung-aspirate cultures were estimated as 97%. Because the literature suggests that these tests are in fact 100% specific,12,13 we decided to constrain these 2 specificities to 100%. The fit of the constrained model did not differ significantly from that of the unconstrained model (likelihood ratio test χ2 = 2.5; P = 0.28 with 2 df).
Table 2 reports the estimates of Model A with 95% confidence intervals (CIs). The goodness-of-fit test, conditional on the MAR assumption,35 indicated that Model A fitted the data well (χ2 = 18.5 with 22 df; P = 0.68). The estimated proportion of S. pneumoniae among inpatients with pneumonia is 0.46 (95% CI = 0.36–0.57).
The estimates of the secondary analysis (Model B), where the sensitivity and specificity estimates for the culture tests were permitted to vary between antibiotic-exposure groups, are shown in Table 3. For patients under antimicrobial treatment, the sensitivities of culture tests are considerably reduced. To illustrate the magnitude of bias, Table 3 also reports the estimates of the Model C where the antibiotic exposure is completely unaccounted for (detailed summaries of Models B and C are available with the online version of this article, http://links.lww.com/EDE/A399).
According to the predicted 276 individual probabilities using Model A, the average probability of S. pneumoniae in those exposed to antibiotics (48%) was slightly higher than in nonexposed (42%). The probability of pneumococcal etiology did not differ in those with a fatal outcome.
We also conducted an analysis similar to Model B, in which we allowed the test accuracies to differ between HIV-infected (n = 147) and HIV noninfected (n = 134); we found no differences (likelihood ratio test against Model A; χ2 = 8.7 with 8 df; P = 0.37). The prevalence estimate, when fitted to a subgroup of HIV-infected only, was 43% (95% CI = 31–55). For HIV noninfected, the model did not identify.
Our primary objective of the study was not to evaluate assays for individual diagnosis but rather to estimate the contribution of S. pneumoniae to the burden of community-acquired pneumonia as accurately as possible. We used a study that included a comprehensive set of laboratory diagnostics together with a latent class analysis that accounted for the bias due to antimicrobial exposure. Among Kenyan adult patients with pneumonia leading to hospitalization, we estimate that 46% of episodes are attributable to pneumococcus.
We found no differences in either pneumococcal prevalence or the test accuracies between HIV-infected and HIV noninfected patients. Although HIV status is an important factor in most health outcomes, our results suggest that HIV is not associated with the ability of the tests to detect pneumococcus as the causative pathogen for pneumonia. In contrast, antimicrobial exposure is a highly important factor when estimating pneumococcal prevalence on the basis of imperfect diagnostic test results.
Of our 3 models, Model A provides the most accurate estimate of the prevalence of S. pneumoniae among patients with pneumonia by eliminating the bias inherent in using culture results from patients who have taken antibiotics. In Model C, where no account is taken of antimicrobial treatment, the prevalence of S. pneumoniae is underestimated by almost 15% as compared with Model A. It is important to note that the test accuracies derived by Model A correspond to a population in which no antimicrobial treatment is used. Because in practice, such a population is highly improbable, we also conducted an analysis to evaluate the test accuracies under antimicrobial treatment (Model B).
As expected, cultures of blood and lung aspirates were highly specific tests for determining pneumococcal involvement. The sensitivity of blood culture (75%) was markedly higher than in a previous report13 (36%), where subjects under antimicrobial treatment were excluded from analysis. However, our analysis showed that treatment by antimicrobials strongly reduces blood-culture sensitivity (24% when exposed to antibiotics), and that the determination of antimicrobial exposure can be considerably biased without biomarker information. Focusing only on those not treated with antibiotics may also introduce selection bias.
A rapid immunochromatographic test (ICT) for C-polysaccharide antigen detection in urine has shown sensitivities in the range of 70%–80% and specificities of 90%–100%.36 Conversely, a study in which latent class analysis was applied,13 estimated that the sensitivity and specificity of the ICT urine test were 77% and 71%, respectively. The detection of capsular antigen has also shown high specificity (98%) in a separate validation study,17 whereas the specificity estimate (82%) from our latent class analysis was less impressive. In validation studies, either imperfect gold standards or other patient groups are used, whereas in latent class analysis, the estimates are based on the concept of true disease. It must be noted, however, that the true disease status in latent class analysis is inherently unverifiable.
A 2-fold increase in Anti-PsaA IgG was the most sensitive diagnostic test (88%). Although the specificity (83%) would not be high enough for the purposes of a vaccine trial, the test may be useful in providing an approximate estimate of the prevalence of pneumococcus in pneumonia patients, especially as the assay is unaffected by antimicrobial treatment. As our study has shown, accounting for the effect of antimicrobial exposure is challenging, both in measurement and in analysis.
Pneumococcal colonization is a prerequisite for pneumococcal pneumonia. However, coincidental carriage during a pneumonia episode is also relatively common and may lead to false positives by other diagnostic tests. Our analysis aimed to separate these 2 processes, both of which originate from pneumococcal colonization. Our view is that the most coherent approach for this separation is through a joint analysis of all test results using latent class analysis. As an indirect proof of the validity of Model A, we derived the positive predictive value of nasopharyngeal swab culture, which was 0.59. Given that pneumococcus had a prevalence of 46% in pneumonia cases, the nasopharyngeal swab culture of pneumococcus adds little to the etiologic diagnosis.
Among diagnostic studies of pneumonia, a positive etiologic test is consistently found among only about two-thirds of patients.5,10,11 It has been suggested that a large proportion of cases that remain undiagnosed are in fact caused by pneumococcus.5 Unfortunately, this suggested verification bias is generally intractable, because none of the currently available tests is sensitive enough to corroborate the hypothesis. It is surprising that recent advances in laboratory science have not been translated into epidemiologic gains in the diagnosis of pneumococcal pneumonia. Polymerase chain reaction (PCR) for pneumococcal gene targets, which may detect numbers of organisms too low to grow in culture,37 and detect nonviable organisms after treatment with antibiotics,33 has shown sensitivities in clinical studies of only 29%–69%38–40 when compared with blood cultures. However, despite all the difficulties and uncertainties in pneumonia diagnosis, our analyses provide at least a lower limit of the disease burden caused by S. pneumoniae.
Among children in the Gambia, a 9-valent pneumococcal conjugate vaccine has been shown to have an efficacy of 50% against invasive pneumococcal disease, and an efficacy of 37% against radiologically confirmed pneumonia.7 The Global Alliance for Vaccine and Immunization has signaled its intention to accelerate introduction of conjugate pneumococcal vaccine for children in the developing world. As radiologically confirmed pneumonia is a common cause of adult morbidity and mortality in sub-Saharan Africa, it is pertinent to ask whether adults would not likewise benefit from immunization. The efficacy of the vaccine is unknown in adults, but our estimate that almost half of inpatient episodes of pneumonia in Kenyan adults are caused by S. pneumoniae provides a strong incentive to investigate it.
We thank all those who contributed to the studies used as source material in this analysis. C. Muyodi and E. Getambu helped to recruit the pneumonia patients in the original etiology study; F. Gleeson reviewed the radiographs; F. Drobniewski, D. Crook, D. Griffiths, K. Mandaliya, B. Chohan, B. Lowe, A. Hannington, M. Ross, and S. Mwarumba helped with the laboratory studies of etiology; A. J. Hall and K. Marsh assisted with the study design of the etiology study and subsequent diagnostic studies; M. Leinonen assisted with the EIA for pneumolysin immune complexes and G. Carlone, J. Sampson, E. Marston, J. Obiero, and J. Whitmon helped with the ELISA for Anti-PsaA IgG and the PCR for PsaA in lung aspirates. This paper is published with the permission of the Director, Kenya Medical Research Institute.
1.Williams EH, Hayes RJ, Smith PG. Admissions to a rural hospital in the West Nile District of Uganda over a 27 year period. J Trop Med Hyg
2.Centers for Disease Control and Prevention. Pneumonia and influenza death rates—United States, 1979–1994. MMWR Morb Mortal Wkly Rep.
3.Macfarlane JT, Adegboye DS, Warrell MJ. Mycoplasma pneumoniae and aetiology of lobar pneumonia in northern Nigeria. Thorax
4.Sow O, Frechet M, Diallo AA, et al. Community acquired pneumonia in adults: a study comparing clinical features and outcome in Africa (Republic of Guinea) and Europe (France). Thorax
5.Scott JA, Hall AJ, Muyodi C, et al. Aetiology, outcome, and risk factors for mortality among adults with acute pneumonia in Kenya. Lancet
6.Mulholland K, Hilton S, Adegbola R, et al. Randomised trial of Haemophilus influenzae type-b tetanus protein conjugate vaccine [corrected] for prevention of pneumonia and meningitis in Gambian infants. Lancet
7.Cutts FT, Zaman SM, Enwere G, et al. Efficacy of nine-valent pneumococcal conjugate vaccine against pneumonia and invasive pneumococcal disease in The Gambia: randomised, double-blind, placebo-controlled trial. Lancet
8.Fedson DS, Scott JA. The burden of pneumococcal disease among adults in developed and developing countries: what is and is not known. Vaccine
9.Koulla-Shiro S, Kuaban C, Belec L. Acute community-acquired bacterial pneumonia in Human Immunodeficiency Virus (HIV) infected and non-HIV-infected adult patients in Cameroon: aetiology and outcome. Tuber Lung Dis
10.Marrie TJ, Durant H, Yates L. Community-acquired pneumonia requiring hospitalization: 5-year prospective study. Rev Infect Dis
11.British Thoracic Society Research Committee, the Public Health Laboratory Service. Community-acquired pneumonia in adults in British hospitals in 1982–1983: A survey of aetiology, mortality, prognostic factors and outcome. Q J Med.
12.Scott JA, Hall AJ. The value and complications of percutaneous transthoracic lung aspiration for the etiologic diagnosis of community acquired pneumonia. Chest
13.Butler JC, Bosshardt SC, Phelan M, et al. Classical and latent class analysis evaluation of sputum polymerase chain reaction and urine antigen testing for diagnosis of pneumococcal pneumonia in adults. J Infect Dis
14.Allen SC. Lobar pneumonia in Northern Zambia: Clinical study of 502 patients. Thorax
15.Koulla-Shiro S, Kuaban C, Auckenthaler R, Belec L, Ngu JL. Adult response to initial treatment with ampicillin in community acquired pneumonia in Yaounde, Cameroon. Cent Afr J Med
16.Yoshimine H, Oishi K, Mubiru F, et al. Community-acquired pneumonia in Ugandan adults: short-term parenteral ampicillin therapy for bacterial pneumonia. Am J Trop Med Hyg
17.Scott JA, Hannington A, Marsh K, Hall AJ. Diagnosis of pneumococcal pneumonia in epidemiological studies: Evaluation in Kenyan adults of a serotype-specific urine latex agglutination assay. Clin Infect Dis
18.Scott JA, Hall AJ, Leinonen M. Validation of immune-complex enzyme immunoassays for diagnosis of pneumococcal pneumonia among adults in Kenya. Clin Diagn Lab Immunol
19.Scott JA, Obiero J, Hall AJ, Marsh K. Validation of immunoglobulin G enzyme-linked immunosorbent assay for antibodies to pneumococcal surface adhesin A in the diagnosis of pneumococcal pneumonia among adults in Kenya. J Infect Dis
20.Scott JA, Marston EL, Hall AJ, Marsh K. Diagnosis of pneumococcal pneumonia by PsaA PCR analysis of lung aspirates from adult patients in Kenya. J Clin Microbiol
21.Scott JA, Hall AJ, Hannington A, et al. Serotype distribution and prevalence of resistance to benzylpenicillin in three representative populations of Streptococcus pneumoniae isolates from the coast of Kenya. Clin Infect Dis
22.Capeding RM, Nohynek H, Ruutu P, Leinonen M. Evaluation of a new tube latex agglutination test for detection of type-specific pneumococcal antigen in urine. J Clin Microbiol
23.Tharpe JA, Russell H, Leinonen M, et al. Comparison of a pneumococcal common protein (PsaA) antibody ELISA and a PsaA immune complex ELISA for detection of pneumococcal serum antibody. Pathobiology
24.Morrison KE, Lake D, Crook J, et al. Confirmation of PsaA in all 90 serotypes of Streptococcus pneumoniae by PCR and potential of this assay for identification and diagnosis. J Clin Microbiol
25.Formann A, Kohlmann T. Latent class analysis in medical research. Stat Methods Med Res
26.Albert PS, McShane LM, Shih JH. Latent Class modeling approaches for assessing diagnostic error without a gold standard: with applications to p53 immunohistochemical assays in bladder tumors. Biometrics
27.Albert PS, Dodd LE. A cautionary note on the robustness of latent class models for estimating diagnostic error without gold standard. Biometrics
28.Hagenaars JA. Latent structure models with direct effects between indicators. Sociol Methods Res
29.Alonzo TA, Pepe MS. Using a combination of reference tests to asses the accuracy of a new diagnostic test. Stat Med
30.Little RJ, Rubin DB. Statistical Analysis with Missing Data.
2nd ed. New York: Wiley-Interscience; 2002.
31.Schafer JL, Graham JW. Missing data: Our view of the state of art. Psychol Methods
32.Collins LM, Schafer JL, Kam CM. A comparison of inclusive and restrictive strategies in modern missing-data procedures. Psychol Methods
33.Wheeler J, Murphy OM, Freeman R, Kearns AM, Steward M, Lee MJ. PCR can add to detection of pneumococcal disease in pneumonic patients receiving antibiotics at admission. J Clin Microbiol
34.Musher DM, Mediwala R, Phan HM, Chen G, Baughn RE. Nonspecificity of assaying for IgG antibody to pneumolysin in circulating immune complexes as a means to diagnose pneumococcal pneumonia. Clin Infect Dis
35.Fuchs C. Maximum likelihood estimation and model selection in contingency tables with missing data. J Am Stat Assoc
36.Roson B, Fernandez-Sabe N, Carratala J, et al. Contribution of a urinary antigen assay (Binax NOW) to the early diagnosis of pneumococcal pneumonia. Clin Infect Dis
37.Rudolph KM, Parkinson AJ, Black CM, Mayer LW. Evaluation of polymerase chain reaction for diagnosis of pneumococcal pneumonia. J Clin Microbiol
38.Lorente ML, Falguera M, Nogues A, Gonzalez AR, Merino MT, Caballero MR. Diagnosis of pneumococcal pneumonia by polymerase chain reaction (PCR) in whole blood: a prospective clinical study. Thorax
39.Murdoch DR, Anderson TP, Beynon KA, et al. Evaluation of a PCR assay for detection of Streptococcus pneumoniae in respiratory and nonrespiratory samples from adults with community-acquired pneumonia. J Clin Microbiol
40.van Haeften R, Palladino S, Kay I, Keil T, Heath C, Waterer GW. A quantitative LightCycler PCR to detect Streptococcus pneumoniae in blood and CSF. Diagn Microbiol Infect Dis