Diagnostic Accuracy of Procalcitonin in Bacterial Meningitis Versus Nonbacterial Meningitis

A Systematic Review and Meta-Analysis

Abstract

Several studies have investigated the diagnostic accuracy of procalcitonin (PCT) levels in blood or cerebrospinal fluid (CSF) in bacterial meningitis (BM), but the results were heterogeneous.

The aim of the present study was to ascertain the diagnostic accuracy of PCT as a marker for BM detection.

A systematic search of the EMBASE, Scopus, Web of Science, and PubMed databases was performed to identify studies published before December 7, 2015 investigating the diagnostic accuracy of PCT for BM. The quality of the eligible studies was assessed using the revised Quality Assessment for Studies of Diagnostic Accuracy method. The overall diagnostic accuracy of PCT detection in CSF or blood was pooled using the bivariate model.

Twenty-two studies involving 2058 subjects were included in this systematic review and meta-analysis. The overall specificities and sensitivities were 0.86 and 0.80 for CSF PCT, and 0.97 and 0.95 for blood PCT, respectively. Areas under the summary receiver operating characteristic curves were 0.90 and 0.98 for CSF PCT and blood PCT, respectively.

The major limitation of this systematic review and meta-analysis was the small number of studies included and the heterogeneous diagnostic thresholds adopted by eligible studies.

Our meta-analysis shows that PCT is a useful biomarker for BM diagnosis.

INTRODUCTION

Acute meningitis (AM) is an extremely severe and life-threatening infection, and early diagnosis and prompt treatment are critically important for AM patients due to the high rates of mortality and morbidity associated with the infection.¹ AM is classified into bacterial meningitis (BM) and nonbacterial meningitis (NBM). Differentiation of BM from NBM is critical for early and prompt intervention for BM patients. Furthermore, differentiation of BM from NBM helps avoid unnecessary hospitalization, antibiotic abuse, and increased medical burden. However, differentiating the 2 forms of AM is challenging because they share many similar clinical symptoms, such as fever and headache.² Positive cerebrospinal fluid (CSF) bacterial culture, Gram staining, or detection of bacterial antigens in the CSF represent the gold standard of clinical testing in BM diagnosis. However, although they have high specificity, the sensitivity is poor. Furthermore, bacterial culture is time-consuming. The serum and CSF markers currently used as supplementary markers in BM diagnosis, such as C-reactive protein, are also characterized by inadequate sensitivity and specificity.^3,4 Therefore, discovery of more sensitive and specific markers for BM is desirable.

Procalcitonin (PCT) is a 116-amino-acid protein that is produced primarily by the C cells of the thyroid gland and secreted from leukocytes in the peripheral blood.⁵ In healthy individuals, PCT is secreted at levels that are below the detectable limit. However, serum PCT levels increase markedly in patients suffering from bacterial infections.⁶ Therefore, elevated PCT levels may serve as useful diagnostic markers for BM.⁷ During the past decades, many studies have investigated the diagnostic accuracy of serum or CSF PCT in BM. However, the results were not unequivocal. Therefore, we performed a systematic review and meta-analysis to ascertain the diagnostic value of serum and CSF PCT in BM.

MATERIAL AND METHODS

Literature Search

Using the search terms “(PCT or procalcitonin) and meningitis”, the authors ZDH and TTW independently searched PubMed, Scopus, Web of Science, and EMBASE to identify eligible studies published before December 7, 2015. Manual searches were also conducted by reviewing the references of the eligible studies. The 2 authors (ZDH and TTW) independently reviewed the titles and abstracts of all studies retrieved by independent searches to identify potentially eligible studies. If necessary, a full-text review was conducted, and any disagreements concerning study selection were resolved by full-text review. Since our work is based on available studies, patient consent was waived.

Inclusion and Exclusion Criteria

Inclusion criteria were: studies that evaluated the diagnostic accuracy of PCT for BM in CSF or blood; sample size of BM or NBM patients greater than 10, to avoid selection bias; and 2 by 2 tables constructed from the reported sensitivity and specificity values. Exclusion criteria were: animal studies; non-English publications; and conference abstracts.

Data Extraction and Quality Assessment

Publication years, national origin, BM or NBM patient status, PCT-testing methods, references, area under the receiver operating characteristic curve (area under curve [AUC]), and PCT detection thresholds were independently extracted in duplicate by ZDH and TTW. A 3rd reviewer resolved any discrepancies or disagreements between the independently extracted datasets. The true-positive (TP), false-positive (FP), false-negative (FN), and true-negative (TN) rates were calculated according to the BM and NBM sample size based on the reported sensitivity and specificity of each study as follows: TP = number of BM patients × sensitivity; FN = number of BM patients × (1 − sensitivity); TN = number of NMB patients × specificity; FP = number of NBM patients × (1 − specificity).

TTW and ZDH independently assessed the eligible studies using the revised Quality Assessment for Studies of Diagnostic Accuracy tool.⁸ The items or domains were labeled as unknown in Quality Assessment for Studies of Diagnostic Accuracy tool if the corresponding design characteristics were not reported. Any disagreement in quality assessment was resolved by consensus.

Statistical Analysis

This meta-analysis was performed and reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines for systematic reviews and meta-analyses (Table S1).⁹ Data for the overall diagnostic sensitivity and specificity of PCT for meningitis were pooled using the bivariate model. The bivariate model uses paired sensitivity and specificity as the starting point of analysis and may represent a more reliable indicator of diagnostic accuracy of an index test in meta-analysis when compared with the traditional summary receiver operating characteristic (SROC) approach, which uses the diagnostic odds ratio (DOR) as the main outcome measure.¹⁰ Additionally, since the bivariate model uses a random effects approach for both specificity and sensitivity, the degree of heterogeneity beyond chance may be attributed to clinical and methodological differences between studies. Pooled positive and negative likelihood ratios were calculated according to the summary estimates of sensitivity and specificity. Funnel plots and Deeks test were used to test for potential publication bias.¹¹ All analyses were performed using Stata 13.0¹² (Stata Corp LP, College Station, TX), and a P value less than 0.05 was considered statistically significant.

RESULTS

Study Eligibility

Twenty-two studies were included in this systematic review and meta-analysis.^13–34 A flowchart of the eligible studies is shown in Supplementary Figure 1, and the characteristics of the studies included in this report are summarized in Table 1. Nine of the included studies were conducted in Asia^{21–23,26–28,32–34} and 11 were conducted in Europe.^{13–20,25,29,30} The sample sizes in each study ranged from 30 to 254, with a total combined sample size of 2058. To evaluate the efficacy of PCT measurement in BM diagnosis, 2 of the studies^30,33 investigated the diagnostic performance of CSF PCT detection, 17 of the studies focused on serum or plasma PCT detection,^{13–15,17–22,24–29,31,32} and 3 of the studies focused on both serum and CSF PCT detection.^16,23,34 Two of the studies enrolled neurosurgery patients,^27,33 8 studies enrolled pediatric patients,^{13,20–22,24,26,29,32} and 9 studies included adult patients.^{14,15,17–19,23,25,31,34} The remaining 3 studies^16,28,30 did not report the demographics of the enrolled patients. The references used for BM diagnosis varied among the included studies. All studies set CSF culture as an item of reference, and a few studies set one or more of the following as additional items of reference: CSF Gram staining, blood culture, CSF antigen test, clinical signs or symptoms, and laboratory findings. Thirteen of the studies used the immunoluminometric assay (ILMA) LUMI test (BRAHMS Diagnostica, Berlin, Germany) to determine PCT,^{13–17,20–24,26,30,34} 2 used commercial VIDAS PCT assays,^27,33 1 used a commercial Elecsys PCT assay,²⁹ 3 used commercial Kryptor PCT assays,^18,19,25 and 2 used commercial Raybiotech PCT assays.^28,31 Fourteen of the studies^{14–18,21,24–28,31,32,34} were prospective and 3 of the studies^20,29,33 were retrospective. The remaining 5 studies^{13,19,22,23,30} did not report whether their data collection was prospective or retrospective.

TABLE 1: Summary of Eligible Studies

Quality Assessment of Eligible Studies

Table 2 lists the quality assessment of eligible studies. The patient selection method is unknown in 6 of the studies,^{16,19,21,22,27,29,30} because the authors failed to report whether the subjects were enrolled consecutively or randomly. The patient selection domain was labeled “high” in 7 studies because healthy individuals were enrolled in the study,^24,28 the study included appropriate exclusion criteria^18,25,31 or the authors mentioned retrospective design.^17,20 The index test domain was labeled “unknown” in 7 studies due to small sample sizes and the lack of a report by the authors indicating whether or not the thresholds were prespecified.^{16,24,26–28,31,32} The index test domain was labeled “high” in 3 studies because the threshold was not prespecified.^14,18,29 The reference standard domain of all eligible studies, except for one,³² was labeled “low” because the reference standard that was used in each eligible study correctly classified the BM and was interpreted without knowledge of the PCT results. The follow-up and timing domains of 6 studies were labeled “unknown” because it was uncertain whether partial verification bias was avoided in those studies.^{14,16,18,19,21–25} The follow-up and timing domains were labeled “high” in 4 studies because not all patients were included in their analysis.^17,20,27,29

TABLE 2: Quality Assessment of Eligible Studies

Diagnostic Accuracy of PCT

Table 3 summarizes the diagnostic accuracy of all eligible studies. The overall diagnostic accuracy was pooled using the bivariate model. A forest plot depicting the diagnostic sensitivity and specificity of blood PCT and CSF PCT detection is illustrated in Figure 1. Overall, the diagnostic sensitivity of CSF PCT detection was 0.80 (95% CI, 0.61–0.91), specificity was 0.86 (95% CI, 0.70–0.95), positive likelihood ratio (PLR) was 5.9 (95% CI, 2.4–14.0), negative likelihood ratios (NLR) was 0.23 (95% CI, 0.12–0.47), and DOR was 25 (95% CI, 8–78). I² across all eligible CSF PCT studies was 0.67 (95% CI, 0.26–1.00), and only 9% of the observed heterogeneity was attributed to the threshold effect. The overall diagnostic sensitivity of blood PCT detection was 0.95 (95% CI, 0.89–0.97), specificity was 0.97 (95% CI, 0.89–0.99), PLR was 31.7 (95% CI, 8.0–124.8), NLR was 0.06 (95% CI, 0.03–0.11), DOR was 568 (95% CI, 103–3141). I² across all eligible studies was 0.96 (95% CI, 0.92–0.99). It is likely that only 27% of the observed heterogeneity was due to the threshold effect.

TABLE 3: Meta-Analysis: Key Findings of Eligible Studies

FIGURE 1: Forest plot of the sensitivity and specificity of PCT for BM diagnosis. BM = bacterial meningitis, PCT = procalcitonin.

The SROC curves for CSF PCT and blood PCT are shown in Figure 2. The AUCs for CSF PCT and blood PCT were 0.90 (95% CI, 0.87–0.92) and 0.98 (95% CI, 0.97–0.99), respectively. The 95% CIs for the AUCs of CSF PCT and blood PCT did not overlap, indicating that the overall diagnostic accuracy of blood PCT detection was superior to CSF PCT.

FIGURE 2: The SROC AUC of PCT in BM diagnosis. The overall diagnostic efficiency of PCT in BM is summarized by the regression curve. AUC = area under curve, BM = bacterial meningitis, PCT = procalcitonin, SROC = summary receiver operating characteristic.

The diagnostic sensitivity, specificity, PLR, NLR, DOR, AUCs for SROCs, I², and proportion of heterogeneity attributed to the threshold effect for blood PCT and CSF PCT are listed in Table 4.

TABLE 4: Overall Diagnostic Characteristics Associated with Blood PCT and CSF PCT

We next analyzed the CSF PCT or blood PCT posttest probability of BM. As shown in Figure 3, the pretest probabilities of BM for blood PCT and CSF PCT were 0.34 and 0.36, respectively. The posttest probabilities of BM after a positive CSF PCT or blood PCT test were 0.77 and 0.94, respectively. The posttest probability values associated with negative CSF PCT or blood PCT tests were 0.12 and 0.03, respectively.

FIGURE 3: Fagan nomogram of the blood PCT and CSF PCT tests for BM diagnosis. BM = bacterial meningitis, CSF = cerebrospinal fluid, PCT = procalcitonin.

Subgroup Analysis and Meta-Regression

Sources of significant heterogeneity between studies investigating blood PCT were determined using subgroup analysis and meta-regression. The type of data collection (prospective or retrospective) was the source of heterogeneity in sensitivity (P < 0.01; Figure 4). In the joint model, none of the study characteristics (data collection, age, test assay, and subject sources) represented sources of heterogeneity.

FIGURE 4: Subgroup analysis of PCT sensitivity and specificity in BM diagnosis. BM = bacterial meningitis, PCT = procalcitonin.

Publication Bias

The funnel plots for publication bias were asymmetrical (Figure 5), suggesting significant publication bias. The statistical significance of this publication bias for both CSF PCT and blood PCT was confirmed using Deeks test (P < 0.05 for both).

FIGURE 5: Funnel plot of potential publication bias. Each solid rectangle in the funnel plot represents an eligible study.

DISCUSSION

The results of this meta-analysis indicate that CSF PCT and blood PCT were both effective biomarkers for BM diagnosis. The diagnostic accuracy of elevated blood PCT appeared to be superior to CSF PCT. Additional bias associated with patient selection and partial verification was the major flaw in the design of the eligible studies. Publication bias existed across all eligible studies.

Two meta-analyses investigated the diagnostic value of PCT for BM.^35,36 Compared with the 2 studies, the strengths of our work are as follows. First, we used a bivariate model instead of a random-effects model to pool the sensitivity and specificity in studies. Therefore, the results of our work are more reliable. Second, previous studies only investigated the diagnostic value of serum PCT for BM, while our study investigated the diagnostic value of both serum and CSF PCT for BM, and therefore, our work is more informative.

We found that blood PCT was associated with a higher pooled sensitivity and specificity when compared with CSF PCT. This finding suggests that blood PCT has superior diagnostic potential when compared with CSF PCT. Furthermore, the superior diagnostic potential of blood PCT was confirmed by SROC analysis, which indicated that the AUCs for CSF PCT were lower than that of blood PCT. Although no single statistical method compared the AUCs of the SROCs, we found no overlap between the 95% CI of the AUCs for CSF PCT and blood PCT, demonstrating that the overall diagnostic accuracy of blood PCT was superior to CSF PCT.

The DOR is an independent indicator of test accuracy that compares the odds of TP patients with the odds of FPs.³⁷ The results of the test range from 0 to infinity, and higher values indicate a better discriminatory test performance.³⁸ The present meta-analysis yielded DOR values of 568 and 25 for blood PCT and CSF PCT, respectively, indicating that both CSF PCT and blood PCT were effective markers for BM diagnosis. Furthermore, the results indicate that the diagnostic accuracy of blood PCT was superior to CSF PCT.

The pooled PLRs and NLRs are more clinically useful than the sensitivity, specificity, DOR, or AUC. Positive likelihood ratios greater than 10 or negative likelihood ratios below 0.1 generate large and often conclusive shifts from pre- to posttest probability (indicating high accuracy). We found that the PLR for blood PCT was 31.7 indicating that patients with BM have an approximately 32-fold higher chance of being PCT positive when compared with BM negative patients. Conversely, we found that the NLR for blood PCT was 0.06, suggesting that a negative blood PCT result was associated with a mere 6% probability that the patient had BM.

The Fagan nomogram also confirmed the extremely high diagnostic accuracy of blood PCT for BM. The BM pretest was approximately 0.36; however, the posttest probabilities associated with positive and negative PCT were 0.94 and 0.03, respectively. The results indicate that the probability of BM was as high as 94% for patients who tested positive for PCT, but only 3% for patients negative for PCT. These results indicate that positive blood PCT can be used to confirm a diagnosis of BM, while negative blood PCT alone is sufficient to rule out BM. The PLR and NLR for CSF were 5.9 and 0.23, respectively, indicating that CSF PCT alone is insufficient to confirm or rule out BM.

The turnaround time for blood PCT or CSF PCT analysis is shorter than that of traditional bacterial culture. Compared with CSF Gram staining, PCT is an objective test that can be reliability implemented independent of laboratory technical expertise. The present meta-analysis revealed a high level of heterogeneity among all the eligible studies, and only a small portion of the heterogeneity was explained by the threshold effect. Meta-regression analysis revealed that the type of data collection (prospective or retrospective), the age of subjects (pediatric or nonpediatric), test assay (Lumi test), and sources of subjects (Asian or other) were not the source of heterogeneity. Further studies are needed to explore the sources of heterogeneity.

Our analysis revealed the following design flaws in the eligible studies:

Some of the eligible studies failed to incorporate inclusion and exclusion criteria.³⁹ Additionally, they also failed to report whether or not the subjects were consecutively enrolled. Because these design flaws resulted in subject populations that may not reflect clinical reality, they introduced a large amount of bias.^8,40

Partial verification bias was not completely ruled out in some of the eligible studies as they usually confirmed the diagnosis of BM using microbiological examination (reference), but failed to report whether other types of meningitis were excluded. Therefore, additional well-designed studies are needed to rigorously assess the diagnostic accuracy of PCT for BM.

Some of the limitations of this meta-analysis are related to the small sample sizes, especially among studies investigating the diagnostic value of CSF PCT. Furthermore, the thresholds in eligible studies were not consistent, which may be due to various PCT assays used in eligible studies. Finally, our analysis indicated the presence of publication bias, indicating that this report may overestimate the diagnostic accuracy of PCT for BM.

In summary, this meta-analysis reveals that both CSF PCT and blood PCT are effective diagnostic markers for BM. However, blood PCT appears to exhibit superior diagnostic accuracy when compared with CSF PCT, and blood PCT alone is sufficient to confirm or exclude BM diagnosis. Additional well-designed studies are needed to corroborate the results of this meta-analysis.

Acknowledgments

The authors thank Medjaden Bioscience Limited, Hong Kong, China, for assisting with the preparation of this manuscript. The authors also thank the study program supported by grants from the National Natural Science Foundation of China (81302541, 81471608).