From the Department of Gynecology, MCP-Hahnemann University, Graduate Hospital, Philadelphia, PA.
Address correspondence to: Amy S. Cooper, Division of Research, Women and Infants’ Hospital of Rhode Island, 101 Dudley Street, Providence, R1 02905; firstname.lastname@example.org
Three new techniques have been developed to improve the quality of cytologic screening for cervical cancer. The U.S. Food and Drug Administration (FDA) has carefully reviewed each of these devices, and the safety and efficacy of each has been confirmed. Two of the techniques [PAPNET Testing System (Neuromedical Systems, Suffern, NY) and AutoPap 300 QC (TriPath Imaging, Burlington, NC)] were approved as rescreening devices that are meant to facilitate the Clinical Laboratories Improvement Act-mandated 10% review of Pap smears that have been interpreted as normal. The third device [ThinPrep (Cytyc Corp, Boxborough, MA)] is a new collection system that is liquid based and that was approved as a replacement for the conventional Pap smear. Since these devices were approved in 1995 and 1996, a considerable effort has been expended to evaluate their performance in a variety of clinical situations. These new technologies have been compared with conventional cytology with regard to false negative rates, sensitivity, and specificity. Recently, attention has been directed to the cost effectiveness of these new technologies and the precise role they should play in modern cytologic screening for cervical cancer. The purpose of this paper is to briefly review the impact of the new technologies with regard to conventional cytologic screening, to review the literature with regard to cost effectiveness, and to suggest how the current dispute might be resolved.
Cervical cancer is the second most common malignancy in women worldwide. In the United States, nearly 14,000 women will develop cervical cancer in 1998 and approximately 60,000 women will be diagnosed with cervical intraepithelial neoplasia (CIN). During her lifetime, an American woman has about a 0.85% chance of developing cervical cancer and about a 0.25% chance of dying from cervical cancer. 1 Fortunately, the incidence of cervical cancer has declined approximately 70% since the introduction of the Pap smear in the late 1940s. Since 1973, the incidence and mortality of cervical cancer has diminished 40%. Despite the fact that no randomized trials ever tested the efficacy of Pap smear screening for cervix cancer, it is one of a select few interventions rated “A” by the U.S. Preventive Services Task Force. 2 This remarkable decrease is a tribute to the diligence of health care providers, cytotechnicians, pathologists, and the lay public. It has been accomplished despite the well-recognized deficiencies of an individual Pap smear. False negative smear rates reported in the literature vary from 5% to 55%. 3 It is difficult to arrive at a precise definition of false negative rates because the definition varies in published reports. The false negative rate is lower when the threshold is CIN 3 or high-grade squamous intraepithelial lesion than when it is atypical squamous (glandular) cells of undetermined significance (ASCUS) or presence of human papillomavirus. In fact, the current practice of annual Pap smears has developed primarily to compensate for the lack of sensitivity of a single Pap smear. Repetitive smears in combination with the slow growth rate of most cervical cancers probably explain the tremendous success of cytologic screening.
Published studies from good academic laboratories assume a false negative rate of 10–15%, but, for the average clinician in the community in which the laboratory provides a wide range of services and in which special training in cytopathology or gynecologic pathology is often minimal, that rate may be as high as 25–40%.
The primary reasons for “false negative” smears are: (1) sampling errors, (2) preparation or technical errors, (3) screening errors, and (4) interpretation errors. The now-standard spatula-and-brush collection technique results in as many as 80% of the collected cells being discarded in the trash with the brush. Errors due to interpretation and screening are felt to account for about 40% of false negatives. Technical and collection errors account for about 60%. 4,5 The new techniques address and overcome these weaknesses. The computer-based methods minimize screening and interpretive errors. The ThinPrep method minimizes sampling and technical errors and, by providing a better-quality smear, diminishes screening and interpretive errors. The new techniques compare very favorably with conventional cytology and diminish the number of false negative smears by increasing sensitivity without affecting specificity. 6
Scientific papers published in peer-review journals have supported each of the new techniques. 7–9 Each of the new techniques focuses on a slightly different defect in the cytology screening system. The AutoPap 300 QC has recently been approved for both primary screening and rescreening. The new technologies have been criticized for detecting primarily low-grade lesions and increasing the clinician’s burden by increasing the number of patients who require further evaluation (sometimes as little as repeating a smear in 4–6 months). The early studies of each method detected predominately ASCUS and low-grade squamous intraepithelial lesion abnormalities. More recent studies indicate fewer minor or equivocal abnormalities detected and an increase in important lesions.
The PAPNET technique was used to evaluate smears that have been interpreted as ASCUS. Twenty per cent of these slides were upgraded, and 8 of 84 smears contained cells consistent with CIN 2 or 3. 10 Recent studies using ThinPrep 2000 found that because the quality of the smear was better, fewer smears were interpreted as ASCUS or AGUS (atypical glandular cells of undetermined significance). Because the method of preparing slides with the ThinPrep technique removes most of the confounding inflammatory debris, mucus and blood, an important reduction was noted of smears labeled “satisfactory but limited by (SBLB)”. 11
We must recognize that all of the published studies on cytologic screening are compromised. They compare one cytologic technique with another or with biopsy of colposcopically suspicious areas. These data are helpful in assessing the positive predictive value of a method. To really measure the true sensitivity and specificity of a method, a certain proportion of the patients with smears read as “normal” would need to at least undergo colposcopy, preferably colposcopy and biopsy, to confirm the absence of disease. To date, no such study exists. With that disclaimer in mind, all three of the new techniques improve the accuracy of cytologic screening without compromising specificity.
Cost Effectiveness Data
Because each of the new techniques adds to the cost of cytologic screening, it is appropriate to assess the “cost effectiveness” of these new techniques. O’Leary et al 12 in a recent article compared automated rescreening techniques with 100% manual rescreening and found the new techniques cost ineffective. This study compared PAPNET rescreening of 5,478 twice-screened negative smears from a low-risk population in the Armed Forces Institute of Pathology with 4,970 once-screened negative smears from the Institute that were then manually rescreened. These are two very different populations. The study design injects a strong bias in favor of manual rescreening. The study population was also too small to achieve statistical significance. The FDA had reviewed effectiveness data of the PAPNET system and prospectively approved the effectiveness claim now published in the product label. The O’Leary article and accompanying commentary by Alan Garber, 12 completely ignored an independent, previously published article by Schechter. 13 Schechter evaluated the PAPNET system and compared it with other established interventions and concluded: “For women screened at moderate frequency, PAPNET testing can reduce morbidity and mortality from cervical cancer and save lives at a cost comparable to that of other widely practiced health care interventions.” Radensky and Mango 14 subsequently published their analysis of the PAPNET system and also concluded that PAPNET was cost effective.
Cytyc Inc, manufacturers of ThinPrep, engaged an independent firm to develop a model to test cost effectiveness of the ThinPrep system. This was similar to a meta-analysis. Incidence figures for the various abnormal smear readings, CIN, and invasive cancer were identified. Costs of interventions (repeat Pap smear, colposcopy, histology, etc) were obtained, and a statistical model was created. A panel of expert gynecologic pathologists and oncologists reviewed this model. Statistics relating to performance of the technique were abstracted from the literature. 15,16 This model demonstrated clear-cut cost effectiveness of the ThinPrep 2000 and did not include any indirect costs such as time lost from work, cost of babysitters, etc. 17
Both ThinPrep and PAPNET, therefore, can support their claims of cost effectiveness with several studies in the literature. AutoPap 300 QC is slightly less expensive than the other two techniques, and should be equally cost effective.
Brown and Garber, 18 in a study supported by the health insurance industry, felt that the new technologies offered only “modest advances” in cancer screening. This analysis was very selective in the articles chosen for comparison and review. The authors chose to select data on an older-model ThinPrep device, which was never used in clinical practice and was succeeded by the model 2000, which was the model reviewed by the FDA. Similarly, early data on the performance of PAPNET were chosen in preference to more recent data with more favorable performance. The performance and price of conventional Pap smears were set at unrealistic levels. The net result of the study design was a strong bias against the new technologies.
The fundamental problem in establishing cost effectiveness in cytologic screening is true failure to use a uniform model of cost assessment. For many years, the article by Eddy 19 has served as the basis for cost comparison in America. This is a flawed model from many points of view. 20 The population on which this study was based is not at all representative of modern American women. Pap smear costs were $3, and the false negative rate of Pap smears was said to be 3%. The Eddy study was never intended to study cost effectiveness but was intended to determine optimal screening intervals. Despite all if its inadequacies, the Eddy model or variations of that model have been the basis for most subsequent studies.
In an effort to prospectively establish a model more reflective of true cost and the true economic impact of screening programs, the U.S. Public Health Service convened a panel of experts in the field of medical ethics, pathology, cost effectiveness, etc, to develop guidelines for future reporting. Their recommendations were published in a series of articles in 1996. 21–23 These recommendations involve calculating events that affect the quality of a woman’s life and the costs associated with these events. Previous models have examined the impact of screening strategies only on the improvement in the length of a woman’s life with no assessment of quality. Certainly, it is important to evaluate how health care strategies impact the length of life. However, with a disease as infrequent as cervical cancer and with cure rates for stage I and II disease as high as they are, only very modest change can occur to improve those numbers. We must remember that in general, a life saved from cervical cancer is a life saved at age 40–50 compared with age 60–70 for heart disease or lung cancer. So the impact economically may be greater because more productive years are saved.
The field of obstetrics and gynecology is committed to practicing evidence-based medicine. Most of our day-to-day clinical judgments are now made on the basis of sound prospective clinical studies. For the last 4 years, studies of the Gynecologic Oncology Group, a federally funded cooperative study group, have contained quality-of-life measurement within most study protocols. When these studies are completed, we will be able to determine the clinical impact as well as the quality-of-life impact of one management option in comparison with another. We are making slow progress in that direction with cytologic screening. The ongoing National Cancer Institute trial will evaluate the natural history of minor cytologic changes as well as clinical management options. This trial will not generate cost effectiveness data comparing the current screening methods. We can at least compare the new technologies with biopsy data now. We must take the next step and compare the performance of new cytologic screening technology with negative and positive results against colposcopy, histology, and the quality of life; only then will we know the true cost effectiveness of these competing technologies.
1. Braly P, Lichter AS, Ash DK, et al
. Draft of Consensus Development Conference Statement, Cervical Cancer. Bethesda: National Institutes of Health, 1996.
2. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services. Baltimore: Williams and Wilkins, 1996.
3. Sprenger E, Schwarzmann P, Kirkpatrick MO, et al
. The false negative rate in cervical cytology: comparison of monolayers to conventional smears. Acta Cytol 1996; 40: 81–89.
4. Goodman A, Hutchinson ML. Cell surplus on sampling devices after routine cervical cytological smears: a study of residual cell populations. J Reprod Med 1996; 41: 239–241.
5. Van der Graf Y, Voojis GP. False negative rate in cervical cytology. J Clin Pathol 1987; 40: 438–442.
6. Lee KR, Ashfaq R, Birdsong GG, Corkill ME, McIntosh KM, Inhorn SL. Comparison of conventional Papanicolaou smears and a fluid based, thin layer system for cervical cancer screening. Obstet Gynecol 1997; 90: 278–284.
7. Papillo JL, Zarka MA, St John TL. Evaluation of the Thin Prep Pap test in clinical practice: a seven-month, 16314-case experience in northern Vermont. Acta Cytol 1998; 42: 203–208.
8. Colgan TJ, Patten SF Jr, Lee JS. A clinical trial of the AutoPap 300 QC system for quality control of cervicovaginal cytology in the clinical laboratory. Acta Cytol. 1995; 39: 1191–1198.
9. Koss LG, Sherman ME, Cohen MB, et al
. Significant reduction in the rate of false negative cervical smears with neural network-based technology (PAPNET Testing System). Hum Pathol 1997; 28: 1196–1203.
10. Kaufman RH, Schreiber K, Carter T. Analysis of atypical squamous (glandular) cells of undetermined significance smears by neural network-directed review. Obstet Gynecol 1998; 91: 556–560.
11. Hutchinson ML, Isenstein LM, Goodman A, et al
. Homogeneous sampling accounts for the increased diagnostic accuracy using the ThinPrep processor. J Clin Pathol 1994; 101: 215–219.
12. O’Leary TJ, Tellardo M, Buckner S, Ali IS, Stevens A, Ollayos CW. PAPNET-assisted rescreening of cervical smears: cost and accuracy compared with a 100% manual rescreening strategy. JAMA 1998; 279: 235–237.
12A. Garber AM. Making the most of Pap testing. JAMA 1998; 279: 240–241.
13. Schecter CB. Cost effectiveness of re-screening conventionally prepared cervical smears by PAPNET testing. Acta Cytol 1996; 40: 1272–1282.
14. Radensky PW, Mango LJ. Interactive neural network-assisted screening: an economic assessment. Acta Cytol 1998; 42: 246–252.
15. Linder J, Zahniser D. The ThinPrep Pap Test: a review of clinical studies. Acta Cytol 1997; 41: 30–38.
16. Roberts JM, Gurley AM, Thurloe JK, Bowditch R, Laverty CRA. Evaluation of the ThinPrep Pap Test as an adjunct to the conventional Pap smear. Med J Aust 1997; 167: 466–469.
17. Cytyc Corp and Quintiles Inc.
18. Brown AD, Garber AM. The Cost Effectiveness of Three New Technologies to Enhance Pap Testing. Chicago: Blue Cross Blue Shield Association, Technical Evaluation Center Assessment Program, Special Assessment, April 1998.
19. Eddy DM. Screening for cervical cancer. Ann Int Med 1990; 113: 214–226.
20. Sedlacek TV, Cooper AS. Cervical cancer screening: what is cost effectiveness? J L Gen Tract Dis 1999; 3 (Suppl 11): S2–S7.
21. Russel LB, Gold MR, Sigel JE, Daniels N, Weinstein MC. The role of cost-effectiveness analysis in health and medicine. JAMA 1996; 276: 1172–1177.
22. Weinstein MC, Sigel JE, Gold MR, Kamlet MS, Russel LB. Recommendations of the panel on cost-effectiveness in health and medicine. JAMA 1996; 276: 1253–1258.
23. Siegel JE, Weinstein MC, Russel LB, Gold MR. Recommendations for reporting cost-effectiveness analyses. JAMA 1996; 276: 1839–1841.
© 2002 Lippincott Williams & Wilkins, Inc.