Wilbur, David C.1; Norton, Mary K.2
The practice of cervical cytology is an inherently subjective and labor-intensive process with a well-documented false-negative rate. Causes of false-negative cervical cytology have been related to many factors, including sampling, specimen preparation, screening, and interpretation errors. Despite the spectacular success of the Papanicolaou screening program in reducing cervical cancer, women continue to die from this disease because of the above-mentioned causes of error in the performance of the examination, and because they fail to have access to, or choose not to participate in, screening programs. 1
Automated devices have been developed that have shown efficacy in the improvement of false-negative rates through the process of quality control (QC) rescreening of cervical cytology smears. 2–4 However, the process of QC rescreening does little to enhance the overall productivity of the cytology laboratory. Such productivity enhancements may become very important in the future. Public health interventions aim to bring additional, previously unscreened, women into the system. Manual screening resources are diminishing, and screening programs are expanding to underdeveloped countries where cervical cancer is still one of the most prevalent causes of cancer mortality.
The AutoPap Primary Screening System was developed to address both goals of improving accuracy and improving productivity in cervical cancer screening. The AutoPap is a computerized scanning device that is designed to be located within the working cytology laboratory. Its major components are described below.
Slide Input and Output
Cervical cytology slides are mounted on trays (eight slides per tray) and are loaded into an input hopper. Slides are scanned automatically and are then offloaded into an output hopper before manual removal from the device. Based on the current throughput of the instrument, approximately 50,000 slides per year can be processed on a single AutoPap device.
High-Speed Video Microscope
This automated microscope is capable of acquiring 25 high-quality images per second. The stage moves continuously, and the static images are obtained “on the fly” by the use of strobe illumination. Focusing is dynamic with three separate video cameras obtaining images at different focal planes, judging which image is optimally focused, and passing it along for further computer analysis.
After acquisition, images are sent to a bank of parallel processing computers designed to analyze entire microscopic fields. The use of parallel processing computers is important to achieve the fastest possible device speed, allowing for large numbers of images to be analyzed in a short period of time. In addition, parallel processing computers are important for device reliability. Failure of one unit does not halt the analysis process; it merely slows it down until the malfunctioning unit can be replaced or repaired.
This portion of the system forms the basis on which the device formulates overall interpretations of each slide. The software consists of separate algorithms for analysis of single cells, cells in groups, and cells in thick groups. Triple algorithmic pathways ensure appropriate analysis of these cytologically different, but common, patterns of cell presentation.
This portion of the software components consists of greater than 20 tests (checks) that ensure that the optical/mechanical apparatus is functioning properly. System integrity checks are run after each tray of eight slides to ensure that the device is operating within acceptable parameters. Any failure will invalidate the results of the preceding eight-slide tray, which must be rerun after investigation of the system-check failure.
Principles of Operation
Before installation of the AutoPap System at any laboratory, TriPath conducts a standard site assessment. A thorough evaluation of slide characteristics for the laboratory takes place. This evaluation ensures that parameters such as stain quality, appropriate coverslipping, etc, are within the operating tolerances of the AutoPap System. This process also allows fixed thresholds for the various output results to be set appropriately for that site’s uniquely processed material. At the present time, “high-risk” specimens (as defined by each individual laboratory, generally including clinical circumstances increasing the patient risk of cervical cancer) are not eligible for the AutoPap primary screening process.
After barcoding of each slide and ensuring that each slide is physically optimal for scanning, slides are ready for processing on the AutoPap System. Each slide undergoes an initial low-magnification scan to map out areas of the slide that contain cellular material and to preselect areas for further high-magnification image acquisition. High magnification is then utilized for obtaining images for analysis.
For each successfully processed slide, the AutoPap generates an internal score from 0 to 1, which indicates the likelihood that an abnormality is present on the given slide. A score of 0 represents low probability of abnormality being present, and a higher score (closer to 1) represents a high probability of abnormality being present. The AutoPap places each slide into one of the following categories, based on the internal score and fixed thresholds.
No Further Review
Such slides can be immediately archived and reported as “within normal limits” without further manual screening. These cases have the lowest probability of being abnormal, as based on the internal score generated for each case (ie, up to 25% of successfully processed slides may be archived and reported as within normal limits without human review.
These slides have a higher probability of containing abnormality than do those in the “no further review” population. All such slides must be screened manually after this designation by AutoPap. In addition to designation as “Review,” the AutoPap provides ranking information derived from the internal score. A slide with a higher probability of having abnormal cells will have a higher ranking than a slide having a lower probability of being abnormal. The “review” slides are stratified into 20% rank increments (quintiles) so that the cytologist will have the information as to where in the risk pool each slide falls.
Quality Control Review
After manual screening, the remaining slides designated as “within normal limits” on the first manual screen, the internal scores of which place them above a separate QC threshold, will require an additional manual screening for QC purposes. These are the slides that have the highest probability of representing false-negatives, and hence this population is the most productive on which to perform QC procedures.
This group of slides contains those that have physical or specimen parameters falling outside the operating limits of the AutoPap (eg, the bar code is unreadable, the coverslip is misplaced, or the slide is not the correct size), and could not be successfully processed on the AutoPap. These slides would be rerun after attempts to repair the slides, or subjected to manual screening. All slides that cannot be successfully processed on the device must be manually screened.
In addition to an internal score and ranking information related to epithelial cell abnormality, the AutoPap also provides information on the successfully processed slides about the specimen adequacy of the slide by assessing the cellularity, both squamous and endocervical, as well as the degree of obscuration of cells by inflammation.
Clinical Trials for Primary Screening
In accordance with the Food and Drug Administration (FDA) requirements and Intersociety Working Group recommendations, NeoPath (now merged into Tripath) sponsored a comprehensive clinical trial in 1997. 5 It included five clinical trial sites, and ultimately more than 25,000 slides were analyzed in the study. The basic design of the study was two armed, directly comparing the results of an “AutoPap-Assisted practice” arm (with 100% AutoPap screening with no greater than a 25% No Further Review population, no less than a 75% Review population, and no less than 15% QC rescreening) with a “Current Practice” arm (100% manual screening and 10% random QC rescreening). The AutoPap-Assisted Practice arm represented the anticipated intended use of the device, whereas the Current Practice arm represented the current standard for non-high-risk slides mandated by the Clinical Laboratory Improvement Amendments of 1988 (CLIA). 6 The study was prospective with cases being randomly selected daily from routine, non-high-risk slides at the trial sites.
All observers in both arms were masked to all other results. Analysis directly compared the results from the Current Practice (manual screening) arm to the AutoPap arm for diagnosis, adequacy, and infectious organism detection.
Unique for cervical cytology state of the art at the time was the truth-determination process used in the trial. To determine true positives and true negatives, a gold standard was required. In the prospective study design, biopsy truth was not immediately available, and when it was available, biopsy confirmation had its own inherent accuracy problems. Because of this problem, guidelines suggested a vigorous cytologic review of all discrepant (and a subsample of concordant) cases. This was performed with three independent cytopathologists. Truth was defined as a consensus diagnosis in two of three independent readings. If no agreement was obtained in the three readings, a multihead microscopy session with three or more cytopathologists ensued until a final cytologic diagnosis could be obtained. This method has since become standard for the industry.
The analysis of results showed that the AutoPap-assisted practice detected more true positive (adjudicated abnormal) cases than did current practice. This improvement held true at all levels of diagnostic abnormality and was statistically superior at the level of all abnormals (atypical squamous cells of undertermined significance and above, ASCUS+) and all squamous intraepithelial lesions low-grade squamous intraepithelial lesions and above, (LSIL+). 7 Statistical equivalency between the arms was demonstrated for high-grade squamous intraepithelial lesions and above (HSIL+). 8 These results directly translate to sensitivity figures for abnormality detection by AutoPap-Assisted Practice and Current Practice arms. For all abnormals (ASCUS+), sensitivity of Current Practice was 79%, whereas AutoPap-Assisted Practice sensitivity was 86%. For LSIL+, Current Practice sensitivity was 84%vs 94% for AutoPap-Assisted Practice. For HSIL+, Current Practice sensitivity was 93%vs 97% for AutoPap-Assisted Practice. 7,8 The AutoPap-Assisted Practice arm sensitivity improvements were not gained at the expense of diminished diagnostic specificity. The AutoPap Practice Arm showed a 16% improvement in diagnostic specificity as compared with the Current Practice arm. 7
Regarding adequacy, there was statistical equivalency between the two arms of the study regarding cases called “satisfactory but limited by....” (SBLB), “unsatisfactory,” and in the detection of “benign cellular changes” (BCC or “reactive cellular changes”) and “infectious organisms.” All of these findings were realized despite the fact that approximately 25% of the cases in the AutoPap-Assisted Practice arm were never manually screened. There was no chance for an abnormal, unsatisfactory, BCC, or infectious organism designation in this population. 9 The AutoPap does report an SBLB result for some slides falling into the No Review population.
On the basis of the data from these clinical trials, the final FDA approval for the AutoPap System reads as follows:10
“The AutoPap Primary Screening System is an automated cervical cytology screening device intended for use in initial screening of Papanicolaou (Pap) smear slides. The AutoPap Primary Screening System identifies up to 25% of successfully processed slides as requiring no further review. The AutoPap Primary Screening System also identifies at least 15% of all successfully processed slides for a second manual review. The device is to be used only on conventionally prepared Pap smear slides and is intended to detect slides with evidence of squamous carcinoma and adenocarcinoma and their usual precursor conditions; it is not intended to be used on slides designated by the laboratory as ”high risk.“ Intended users are trained cytology laboratory personnel operating under the direct supervision of a qualified cytology supervisor or laboratory manager/director.”
In conclusion, the clinical trials of the AutoPap Primary Screening System showed improved diagnostic sensitivity and specificity over the current standard of care in cervical cytology. In addition, based on the No Further Review population of slides, which by definition never need to be screened manually, the possibility exists for substantive productivity enhancements in the cytology laboratory.
Initial independent studies appearing recently in the literature lend support to the results of the clinical trial in actual practice. Bibbo and Hawthorne 11 reported only minor cytologic abnormalities being present within the No Further Review population, consistent with the clinical trial results. In a separate study, Bibbo et al12 reported positive case ranking results consistent with the data from the clinical trials.
It is important to note that the applications discussed below are not approved for use or available to the United States market and are therefore only investigational at this time.
Introduction of Liquid-Based Specimens for Automated Screening
Clinical trials took place in 1999 for the integration of liquid-based specimens 13,14 (TriPath Prep®) with the AutoPap Primary Screening System. Data from this trial have been submitted to the FDA for review. In addition, preliminary data have already shown feasibility for merging Cytyc’s (Boxborough, MA) ThinPrep® Pap Test with the AutoPap System. 13
Introduction of “Location-Guided Screening”
Embedded in the primary screener software is the System’s ability to identify the locations on the slide of the cells most likely to be abnormal on each slide. 15 This functionality is available and being used internationally at the present time with a concept referred to as PAPMAP®. This enhancement is not approved for commercial use in the United States. With the PAPMAP, a printout is made for each slide in the Review population after primary screening on the device. On this printout are shown the circled locations of cellular material of interest, including potentially abnormal cells. These circles can be easily traced onto the glass slides, essentially sending to the cytologist a “predotted” slide. Eventually this transfer is anticipated to be made electronically directly into a computerized microscopy workstation.
Use of this screening enhancement to the AutoPap System holds the potential for further improvements in accuracy and productivity over the current status.
1. Koss LG. The Papanicolaou test for cervical cancer detection: a triumph and a tragedy. JAMA 1989; 261: 737–743.
2. Patten SF, Lee JSJ, Wilbur DC, et al. The AutoPap 300 QC System multicenter clinical trials for use in quality control rescreening of cervical smears. I. Prospective intended use study. Cancer (Cancer Cytopathol) 1997; 81: 343–347.
3. Patten SF, Lee JSJ, Wilbur DC, et al. The AutoPap 300 QC System multicenter clinical trials for use in quality control rescreening of cervical smears. II. Prospective and archival sensitivity studies. Cancer (Cancer Cytopathol) 1997; 81: 337–342.
4. Fetterman B, Pawlick G, Koo H, Hartinger J, Gilbert C, Connell S. Determining the utility and effectiveness of the NeoPath AutoPap 300 QC System used routinely. Acta Cytol 1999; 43: 13–22.
5. Intersociety Working Group for Cytology Technologies. Proposed guidelines for primary screening instruments for gynecologic cytology. Acta Cytol 1997; 41: 924–934.
6. Medicare, Medicaid, and CLIA programs: extension of certain effective dates for clinical laboratory requirements under CLIA. 42 CFR Part 493. Fed Reg Oct 14, 1998;63:55031–55034.
7. Wilbur DC, Prey MU, Miller WM, Pawlick GF, Colgan TJ. The AutoPap System for primary screening in cervical cytology: comparing the results of a prospective, intended-use study with routine manual practice. Acta Cytol 1998; 42: 214–220.
8. Wilbur DC, Prey MU, Miller WM, Pawlick GF, Colgan TJ. AutoPap System detection of high grade squamous intraepithelial lesions and cancers: primary screener clinical trial results. Cancer (Cancer Cytopathol) 1999; 87: 354–358.
9. Wilbur DC, Prey MU, Miller WM, Pawlick GF, Colgan TJ. AutoPap System detection of benign cellular changes and infectious agents: results from the primary screener clinical trials. Diagn Cytopathol 1999; 21: 355–358.
10. Food and Drug Administration. Second Supplement to the Premarket Approval for the AutoPap Device. Pub. No. P950009. Rockville, MD: Food and Drug Administration, 1998.
11. Bibbo M, Hawthorne C. Performance of the AutoPap primary screening system at Jefferson University Hospital. Acta Cytol 1999; 43: 27–29.
12. Bibbo M, Hawthorne C, Zimmerman B. Does use of the AutoPap assisted primary screener improve cytologic diagnosis? Acta Cytol 1999; 43: 23–26.
13. Lee J, Kuan L, Ellison D, et al. Evaluation of the NeoPath AutoPap 300 QC System on Cytyc ThinPrep-prepared slides. Acta Cytol 1996; 40: 1043–1044.
14. Lee J, Bannister W, Ellison DG, Patten FW. Evaluation of the NeoPath AutoPap QC System on AutoCyte-Prep slides. Acta Cytol 1998; 42: 1253.
15. Lee J, Kuan L, Oh S, Patten FW, Wilbur DC. A feasibility study of the AutoPap System location-guided screening. Acta Cytol 1998; 42: 221–226.
© 2002 Lippincott Williams & Wilkins, Inc.