Siebers, Albertus G. MSc1; Klinkhamer, Paul J.J.M. MD2; Arbyn, Marc MD, MSc3; Raifu, Amidu O. MSc3; Massuger, Leon F.A.G. MD, PhD4; Bulten, Johan MD, PhD1
Although successful in reducing the incidence of and mortality from cervical carcinoma, the diagnostic accuracy of screening with conventional Pap tests is hampered by the occurrence of both false-negative and false-positive results. Besides sampling issues during test taking, erroneous results are in great part due to problems with sample preparation and cytologic interpretation. Liquid-based cytology has been developed to address these issues.1–3
Numerous studies have been done comparing the performance of liquid-based cytology with conventional cervical cytology; however, these studies resulted in substantial controversy about whether liquid-based cytology performs better than conventional cytology. Although most studies reported an increased detection of squamous intraepithelial lesions (SIL) and decreased inadequacy rates, several systematic reviews yielded contradictory results depending on the choice of the outcome measure and selection criteria for inclusion of individual studies.4–11
We initiated a large-scale population-based cluster randomized controlled trial (RCT), including almost 90,000 cases. The objective was to prospectively test the cytologic test positivity rates of atypical squamous cells of undetermined significance or more severe (ASCUS+), low-grade squamous intraepithelial lesions or more severe (LSIL+), and high-grade squamous intraepithelial lesions or more severe (HSIL+) of the ThinPrep system (using the ThinPrep 3000 Processor, Cytyc Corporation, Boxborough, MA) in comparison with conventional cervical cytology. For practical reasons, we used family practices as unit of randomization in the cluster design. This report presents the baseline outcomes in terms of odds ratio (OR) for the cytologic test positivity rates of ASCUS+, LSIL+, and HSIL+, taking cluster design into account and applying a per-protocol analysis.
MATERIALS AND METHODS
The randomized controlled trial was performed within the framework of the national cervical screening program in two regions in the Netherlands, in collaboration with local gynecologists, pathologists, and family physicians. The screening program invites women aged 30–60 years every 5 years to have a Pap test done by a family physician. Two clinical laboratory sites (PAMM Laboratories, Eindhoven, and Radboud University Nijmegen Medical Centre, Nijmegen) participated in the trial. All family practices feeding the study sites were eligible for random assignment to the experimental arm (preparation of the test using the liquid-based system) or control arm (preparation of the test using conventional cervical test preparation). Women who were visiting their family practice for participation in the national cervical screening program were all included in the study and received a conventional Pap test or a liquid-based sample according to the random allocation of their respective family practice. Ethical approval for this study was obtained from the Dutch Ministry of Health, Welfare and Sport.
The sample size for this study was calculated based on the baseline assumption of 0.6% HSIL+ in the participants and liquid-based cytology detection of a 33% increase in cervical intraepithelial neoplasia 2 at α=5% and β=20%. With these parameters, we initially computed the sample size of 28,269 by ignoring the clustering of women within practices. To account for the clustering effect, we assumed from the previous routine data from the two sites, an intraclass correlation coefficient of 0.05, with average cluster size of 250 and standard deviation of 200. This led us to the coefficient of variation of 0.8 and a design effect of 1.59.12 By multiplying the design effect by sample size without clustering effect, we obtained a sample size of 44,947 women to be screened in each arm.
The inclusion of 89,960 women screened started in April 2003 and was completed in July 2006. One hundred seventy-six participants were excluded from analysis because their general practitioner was not randomly assigned. Identification data, clinical data, and the screening results of the remaining 89,784 participants were stored in the local pathology databases.
Allocation was based on clusters rather than on individuals, with family practice as the unit of randomization. This was done to prevent contamination by patient preference (selection bias) and for other practical reasons. All practices connected to the two study sites were ranked by postal code, and subsequently, the codes 0 (conventional) or 1 (liquid-based) were allocated using a binomial random number generator.13 The family practices in the catchment areas of the two study sites were stratified by level of urbanization (high urbanization meaning an urban area with more than 100,000 inhabitants) by sorting on postal code. They were assigned to one of the study arms by assigning them at random to conventional or liquid-based screening by the study database manager. All practices participated in the randomization procedure and agreed with the outcome of randomization after being informed by mail. Family practices allocated to the experimental arm were provided with material for test taking with the liquid-based system. Practices allocated to the control arm were provided with the conventional test-taking material. Adherence to the assignment was checked periodically during the study. For obvious reasons blinding for the method could not be realized for sample taking and test reading.
Family physicians or their assistant took the cervical samples. At the start of the trial, all family practices were informed about the study and consented to participation. Next, the practices that converted to liquid-based cytology received additional training, either by a regional course or by in-practice training by the manufacturer.
All cervical samples were obtained using the Rovers Cervex-Brush (Rovers Medical Devices BV, Oss, the Netherlands). Conventional tests were prepared in the usual way, whereas liquid-based cytology users were instructed to rinse their cell samples in PreservCyt (Cytyc Corporation) transport medium according to the manufacturer's instructions by rotating the brush in the solution 10 times while pushing against the PreservCyt vial wall.1 At the laboratory, liquid-based samples were prepared using the ThinPrep 3000 Processor.
At the start of the trial, one of the participating laboratories had experience with screening liquid-based slides for 1 year; the other laboratory did not have previous experience with liquid-based cytology. Before implementation of the liquid-based method in the laboratories, cytotechnologists and cytopathologists attended a 3-day training course, provided by the manufacturer. The course finished with a test, which was mandatory before starting to screen liquid-based cytology slides. During the learning stage a minimum of 200 liquid-based slides, taken from the routine workload, were screened within a multiple screening protocol by two cytotechnologists until cytologic consensus was reached. After these 200 liquid-based slides, cytotechnologists had a final test, and when they passed they were allowed to screen liquid-based cytology independently. Technical operators received instruction for operating and maintenance of the ThinPrep 3000 Processor from Cytyc Corporation.
Both liquid-based and conventional slides were randomly examined by the trained cytotechnology staff and routinely reported using the Dutch CISOE-A classification, which can be translated to the Bethesda 1991 subcategories (ASCUS/AGUS, LSIL, and HSIL).14,15 Abnormal slides with diagnosis HSIL+ were reviewed by a senior cytotechnologist and a trained pathologist as were slides with diagnosis ASCUS/AGUS/LSIL, with an advice for referral to a gynecologist. Cases of ASCUS/AGUS/LSIL with repeat advice followed a multiple screening protocol, with review by a senior cytotechnologist.
Cytologic diagnoses were categorized in four diagnostic categories:
1. Normal (including benign cellular change)
3. LSIL (low-grade intraepithelial squamous lesions with addition of low-grade glandular lesions)
4. HSIL/carcinoma (high-grade intraepithelial squamous lesions or squamous cell carcinoma with addition of adenocarcinoma in situ and cervical adenocarcinoma)
All participants from the randomized practices were included in an intention-to-treat analysis. Only those participants who had the proper test (ie, the study arm their family practice had been assigned to by randomization) were included in the per-protocol analysis. Proportions were compared by using χ2 tests, whereas continuous variables were compared by Student t test. The test positivity rates of the experimental (liquid-based cytology) arm relative to the control arm were assessed for the cytologic outcome of ASCUS, LSIL, ASCUS+ (ASCUS, LSIL, HSIL, and carcinoma), LSIL+ (LSIL, HSIL, and carcinoma) and HSIL+ (HSIL and carcinoma), taking intracluster coefficients into account for assessment of the confidence intervals. Additionally, unsatisfactory rates were analyzed.
Crude and adjusted (controlling for age, urbanization level, study period [defined as first and second half of the study, using the median preparation date as separator] and clinical laboratory site) odds ratios (ORs) for cytologic outcomes were computed using univariable and multivariable logistic regression analysis, also taking the cluster design into account. The number needed to screen was computed as the reciprocal of the risk difference (1/(rateliquid-based–rateconventional). Analyses were performed with SPSS 14.0.2 (SPSS Inc., Chicago, IL) and Stata 9.2 (StataCorp LP, College Station, TX) software.
As shown in Figure 1 and Table 1, there were 89,784 participants, recruited from 246 practices included in the intention-to-treat analysis and 85,076 participants from 246 practices in per-protocol analysis. The number of practices was evenly distributed over the two study arms (122 in the experimental arm and 124 in the control arm). Nevertheless, the overall distribution of individuals between the two study arms was unbalanced, with more samples examined in the experimental (liquid-based cytology) arm (n=49,222) than in the control arm (n=40,562). This was mainly caused by an uneven distribution of liquid-based and conventional slides at site 1 (PAMM laboratory) (57.7% liquid-based compared with 42.3% conventional), due to allocation, by chance, of six large (n>1,000) practices to liquid-based compared with only one to the control arm. The largest clinical laboratory (site 1) examined almost twice the number of slides (57,045) as compared with site 2 (32,739). In site 1, proportion of liquid-based cytology preparation was similar in high-urbanization areas as compared with low-urbanization areas (site 1 was 57.9% liquid-based in high-urbanization compared with 57.5% in low-urbanization areas; P=.37). In site 2, more liquid-based preparations were processed from practices in high-urbanization areas (52.3% liquid-based in high-urbanization areas and 48.9% in low-urbanization areas, P<.001). Women aged younger than 45 years were relatively more often examined with the experimental method (55.8% liquid-based cytology) as compared with women aged 45 years or older (53.7% liquid-based cytology).
The crude ORs, taking the cluster effect into account, for the various cytologic diagnostic categories are shown in Table 2. Only women with a satisfactory index test were included for calculation of proportions of test positivity. The ratios of the odds for test positivity of liquid-based compared with conventional cytology were never significantly different from unity. In contrast, the crude OR of the unsatisfactory rate was 0.30 (95% confidence interval 0.23–0.38), indicating that in the experimental arm, significantly fewer tests were classified as unsatisfactory as compared with the control arm. We also performed an intention-to-treat analysis on the data set but this did not change the results.
As shown in the flow diagram (Fig. 1), test positivity rates of the various cytologic categories varied significantly with the study site (P<.001) as well as with level of urbanization (P<.001). Test positivity rates were higher for all three cytologic cutoffs in study site 2. The same was seen for high-urbanization level, both in study site 1 as well as study site 2. The odds ratios for cytologic abnormalities never differed significantly from unity. These findings did not vary significantly by laboratory, urbanization, or study period (data not shown).
To adjust for potentially confounding variables (age, site, urbanization level, and experience with liquid-based cytology) we used logistic regression. Table 3 provides the crude ORs as well as adjusted ORs (adjusted for differences in age, study site, study period, and urbanization level). Again, none of the diagnostic categories showed a significant difference between the two study arms. The unsatisfactory rate in the liquid-based cytology arm, however, remained significantly lower as compared with the unsatisfactory rate in the control arm (OR 0.29, 95% confidence interval [CI] 0.23–0.38). The number needed to screen to observe an additional cervical abnormality was not statistically significantly different from zero. Per 128 women screened with liquid-based cytology, one unsatisfactory preparation is avoided (number needed to screen –128, 95% CI –111 to –151).
In this large-scale, population-based, cluster randomized controlled trial including almost 90,000 cases, we found no difference in performance between the liquid-based method (experimental arm) and conventional cytology (control arm) in terms of cytologic test positivity rates for the various cutoff points. The cluster randomization of practices resulted in unequal numbers of subjects in the two arms. The overrepresentation of cases in the experimental arm in clinical laboratory site 1 was caused by some large centers of family practices that had been assigned to the experimental arm. These centers were operating in a high-urbanization area that resulted in an overrepresentation of liquid-based tests in this stratum. Potential confounding, due to unequal distribution of factors and the clustering, was controlled for by logistic regression with and without correction for design effect.
Neither the crude nor the adjusted ORs were found to differ significantly from unity in the per-protocol analysis, suggesting that the test positivity rates of liquid-based cytology are similar to conventional cytology. On the other hand, we found a strong reduction in unsatisfactory rates in the experimental liquid-based arm as compared with conventional cytology (OR 0.29, 95% confidence interval 0.23–0.38). Applying an intention-to-treat analysis on the data set did not change results, indicating that the per-protocol analysis did not alter the outcome.
There were striking differences in test positivity rates between the two participating clinical laboratory sites as well as between women living in low- and high-urbanization areas. The difference in test positivity rate between the study sites may reflect differences in cytologic interpretation of the laboratory, but may also be the result of differences in the prevalence of cervical abnormalities. The relation we found between urbanization level and the prevalence of abnormalities of the squamous and glandular epithelium corroborates the results obtained by other investigators16: the higher the urbanization level the higher the prevalence of cervical epithelial lesions. To evaluate a potential learning effect for liquid-based cytology, we analyzed the results from the first half of the trial as well as the second half, but we did not find a significant effect on the ORs.
Most previously performed studies used a split sample design. Although looking perfectly controlled, this study design has raised concerns with respect to a possible disadvantage for liquid-based cytology when the collected cellular material is split, with a conventional test made first and the residual material immersed in the fixative solution.5 Studies using a two-cohort design (in which conventional tests and liquid-based samples are taken from women belonging to separate but similar populations) frequently found higher test positivity rates for liquid-based cytology.17–23 In contrast, we found no difference in test positivity rate between liquid-based and conventional tests, irrespective of the diagnostic cutoff value. Whereas we used a randomized study design, the other studies compared cytologic detection rates with historical cohorts. Most of these studies reported a substantial and statistically significant increase in cytologically detected abnormalities for liquid-based cytology, with the most impressive increase found in screening centers with low rates of abnormalities.20,24 The present study was also performed in a low-risk screening population, but we did not find higher detection rates with liquid-based cytology. The higher detection rates reported with the liquid-based technique in other studies may be caused by the introduction of the liquid-based technique, creating a higher awareness and enthusiasm for the new technique (intention bias). Also, improved quality control, coinciding with the introduction of the new technique, may have resulted in an increased detection of cytologic abnormalities.8 Finally, when using historical data as a control group, differences in the study populations may have biased the results. On the other hand, it may also be the case that the quality of conventional screening in the Netherlands is so high that introduction of the new technique has little additional value.
Only two other randomized controlled trials have been published.25,26 The study from Obwegeser25 was unpowered (n=1,999) and found no difference in test positivity rates between liquid-based and conventional cytology. Ronco et al26 found a significantly higher test positivity rate for liquid-based cytology as compared with conventional cytology (relative frequency 1.57, 95% CI 1.13–2.18). However, this higher test positivity rate in liquid-based cytology was at the expense of a reduced positive predictive value.
Several other studies found higher rates of LSIL and lower rates of ASCUS/AGUS.11,16–18,20 This observation was not found in the present study because both ASCUS and LSIL detection rates did not differ significantly between the liquid-based and conventional study arm.
We did find significantly lower unsatisfactory rates when using liquid-based cytology as preparation technique, which will be advantageous in settings with high proportions of unsatisfactory tests. However, in the Netherlands the unsatisfactory rate for conventional tests is already very low, which reduces the added value of liquid-based cytology in terms of absolute reduction of the number of unsatisfactory tests. Use of the liquid-based method results in this study in a reduction of unsatisfactory tests of 8 per 1,000 tests.
A clear additional benefit of the liquid-based method is the availability of residual material for human papillomavirus reflex testing in case of ASCUS or LSIL.3,27 However, presently, negative triage of ASCUS and LSIL in the Netherlands is not allowed on program tests but only for the follow-up tests of borderline and low-grade program tests.
The present study does not yet allow the conclusion that the diagnostic accuracy of liquid-based and conventional cytology is equal with respect to histologically defined outcomes. It may be theoretically possible that liquid-based cytology would be more sensitive for cervical intraepithelial neoplasia and that the conventional Pap test is less specific or vice versa. Therefore, for definite conclusions, comparison with a blindly verified reference standard is needed to assess the relative sensitivity and positive predictive value for histologically confirmed cervical intraepithelial neoplasia and cancer. These results will be available after completion of the follow-up period and be the subject of a future report.
Our conclusions are that both methods perform equally well in terms of test positivity rates within the setting of the Dutch cervical screening program. The liquid-based method does result in fewer unsatisfactory tests, but in the framework of the Netherlands cervical screening program, this adds little extra because unsatisfactory rates for conventional screening are already very low. However, the liquid-based technique does offer other additional advantages such as availability of material for reflex human papillomavirus testing and other molecular tests.
1. Arbyn M, Herbert A, Schenck U, Nieminen P, Jordan J, McGoogan E, et al. European guidelines for quality assurance in cervical cancer screening: recommendations for collecting samples for conventional and liquid-based cytology. Cytopathology 2007;18:133–9.
2. Hutchinson ML, Isenstein LM, Goodman A, Hurley AA, Douglass KL, Mui KK, et al. Homogeneous sampling accounts for the increased diagnostic accuracy using the ThinPrep Processor. Am J Clin Path 1994;101:215–9.
3. Arbyn M, Buntinx F, Van Ranst M, Paraskevaidis E, Martin-Hirsch P, Dillner J. Virologic versus cytologic triage of women with equivocal Pap smears: a meta-analysis of the accuracy to detect high-grade intraepithelial neoplasia. J Natl Cancer Inst 2004;96:280–93.
4. Davey E, Barratt A, Irwig L, Chan SF, Macaskill P, Mannes P, et al. Effect of study design and quality on unsatisfactory rates, cytology classifications, and accuracy in liquid-based versus conventional cervical cytology: a systematic review. Lancet 2006;367:122–32.
5. Arbyn M, Bergeron C, Klinkhamer P, Martin-Hirsch P, Siebers AG, Bulten J. Liquid compared with conventional cervical cytology: a systematic review and meta-analysis. Obstet Gynecol 2008;111:167–77.
6. Klinkhamer PJ, Meerding WJ, Rosier PF, Hanselaar AG. Liquid based cervical cytology. Cancer 2003;99:263–71.
7. Abulafia O, Pezzullo JC, Sherer DM. Performance of ThinPrep liquid-based cervical cytology in comparison with conventionally prepared Papanicolaou smears: a quantitative survey. Gynecol Oncol 2003;90:137–44.
8. Herbert A, Johnson J. Personal view. Is it reality or an illusion that liquid-based cytology is better than conventional cervical smears? Cytopathology 2001;12:383–9.
9. Hartmann KE, Nanda K, Hall S, Myers E. Technologic advances for evaluation of cervical cytology: is newer better? Obstet Gynecol Surv 2001;56:765–74.
10. Bernstein SJ, Sanchez-Ramos L, Ndubisi B. Liquid-based cervical cytologic smear study and conventional Papanicolaou smears: A metaanalysis of prospective studies comparing cytologic diagnosis and sample adequacy. Am J Obstet Gynecol 2001;185:308–17.
11. Nanda K, McCrory DC, Myers ER, Bastian LA, Hasselblad V, Hickey JD, et al. Accuracy of the Papanicolaou test in screening for and follow-up of cervical cytologic abnormalities: a systematic review. Ann Intern Med 2000;132:810–9.
12. Eldridge SM, Ashby D, Kerry S. Sample size for cluster randomized trials: effect of coefficient of variation of cluster size and analysis method. Int J Epidemiol 2006;35:1292–300.
13. Hilbe J, Linde-Zwirble W. Random number generators. Stata Tech Bull 1995;28:20–1.
14. The revised Bethesda system for reporting cervical/vaginal cytological diagnoses: report of the 1991 Bethesda Workshop. Acta Cytol 1992;36:273–6.
15. Hanselaar AGJM, Doornewaard H, Noorduyn LA, Weltevreden EF. Praktijkrichtlijn voor het opzetten van een kwaliteitssysteem voor cytopathologisch onderzoek van de baarmoederhals. Nijmegen (the Netherlands): NVVP; 1996.
16. Hatch KD, Sheets E, Kennedy A, Ferris DG, Darragh T, Twiggs L. Multicenter direct to vial evaluation of a liquid-based pap test. J Low Genit Tract Dis 2004;8:308–12.
17. Dupree WB, Suprun HZ, Beckwith DG, Shane JJ, Lucente V. The promise and risk of a new technology: The Lehigh Valley Hospital's experience with liquid-based cervical cytology. Cancer 1998;84:202–7.
18. Bolick DR, Hellman DJ. Laboratory implementation and efficacy assessment of the ThinPrep cervical cancer screening system. Acta Cytol 1998;42:209–13.
19. Papillo JL, Zarka MA, St John TL. Evaluation of the ThinPrep Pap test in clinical practice. A seven-month, 16,314-case experience in northern Vermont. Acta Cytol 1998;42:203–8.
20. Carpenter AB, Davey DD. ThinPrep Pap Test: performance and biopsy follow-up in a university hospital. Cancer 1999;87:105–12.
21. Diaz-Rosario LA, Kabawat SE. Performance of a fluid-based, thin-layer papanicolaou smear method in the clinical setting of an independent laboratory and an outpatient screening population in New England. Arch Pathol Lab Med 1999;123:817–21.
22. Guidos BJ, Selvaggi SM. Use of the Thin Prep Pap Test in clinical practice. Diagn Cytopathol 1999;20:70–73.
23. Weintraub J, Morabia A. Efficacy of a liquid-based thin layer method for cervical cancer screening in a population with a low incidence of cervical cancer. Diagn Cytopathol 2000;22:52–9.
24. Lee KR, Ashfaq R, Birdsong GG, Corkill ME, McIntosh KM, Inhorn SL. Comparison of conventional Papanicolaou smears and a fluid-based, thin-layer system for cervical cancer screening. Obstet Gynecol 1997;90:278–84.
25. Obwegeser JH, Brack S. Does liquid-based technology really improve detection of cervical neoplasia? A prospective, randomized trial comparing the ThinPrep Pap Test with the conventional Pap Test, including follow-up of HSIL cases. Acta Cytol 2001;45:709–14.
26. Ronco G, Cuzick J, Pierotti P, Cariaggi MP, Dalla Palma P, Naldoni C, et al. Accuracy of liquid based versus conventional cytology: overall results of new technologies for cervical cancer screening: randomised controlled trial. BMJ 2007;335:28.
27. Arbyn M, Sasieni P, Meijer CJ, Clavel C, Koliopoulos G, Dillner J. Chapter 9: Clinical applications of HPV testing: a summary of meta-analyses. Vaccine 2006;24 suppl:78–89.