Thomas Sedlacek, a former member of the U.S. Food and Drug Administration (FDA) advisory panel for obstetric and gynecologic medical devices, presented his own cost-benefit analysis of three examples of new diagnostic medical devices that had been approved recently by the FDA for marketing in the United States. These devices, PAPNET (Neuromedical Systems, Suffern, NY), AutoPap 300 QC (TriPath Imaging, Burlington, NC), and Cytyc ThinPrep Processor (Cytyc Corp, Boxborough, MA), were designed and intended to improve the sensitivity of cervical cytology either by obtaining a more representative sample of cervical cells from the cervix or by applying computer-assisted image analysis without decreasing the specificity of cervical cytology for the screening and diagnosis of cervical cancer and its precursors. Dr. Sedlacek provided a historical overview of these devices, their regulatory history, his review of the medical literature on studies of the clinical utility of these devices, and how he applied these data to assess the costs and benefits of the three devices.
The FDA’s mandate for the premarket regulation of medical devices comes from the U.S. Congress. The FDA is the federal executive agency that must assure that there is evidence of reasonable safety and effectiveness of the device for its intended use before it can be distributed commercially for patient diagnosis and management.
“A device is considered safe when it can be determined, based upon valid scientific evidence, that the probable benefits to health from use of the device for its intended uses and conditions of use (indications), when accompanied by adequate directions and warnings against unsafe use, outweigh any probable risks. A device is effective when it can be determined, based upon valid scientific evidence, that in a significant portion of the target population, the use of the device for its intended uses and conditions of use, when accompanied by adequate directions for use and warnings against unsafe use, will provide clinically significant results.”1,2
The Quality Systems Regulations of the FDA apply to the manufacturers’ design, development, manufacturing, and promotion of medical devices. Once a device is approved or cleared, the FDA does not regulate the professional or lay users of medical devices. The FDA does not have the mandate to review the monetary costs or societal costs of medical devices, nor to do technology assessments. By congressional statute, FDA reviews must adhere to strict timelines to ensure that the agency’s regulation does not impede the availability of safe and effective medical devices to the public.
Cost-benefit analyses and technology assessments are authorized or performed by other agencies within the U.S. Department of Health and Human Services such as the Agency for Healthcare Quality and Research, the Centers for Medicare and Medicaid Services, and the Centers for Disease Control and Prevention (CDC); by nongovernment health care providers such as insurance companies; and by independent health professionals.
Thornbury and Fryback developed a six-level hierarchical model for technology assessment and cost-benefit analysis of diagnostic medical devices. Their model ranks the levels of information necessary to assess a medical device:
(1) Technical efficacy: the technical specifications.
(2) Diagnostic accuracy efficacy: the analytic performance and the clinical performance in a test population.
(3) Diagnostic thinking efficacy: the influence of the test result on the differential diagnosis of the clinician.
(4) Therapeutic efficacy: the influence of the test result on the selection of therapy and management of the patient.
(5) Patient outcome efficacy: the benefit of the test result to the clinical outcome of the individual patients, eg, quality of life, morbidity, and mortality.
(6) Societal efficacy: the benefit of the test results to society weighed against other technologies and requirements of society as a whole.
FDA regulation does not extend to societal efficacy. 3
Two of the devices presented by Sedlacek were computer-assisted image-analysis systems that are no longer in commercial distribution. One, the PAPNET, automatically screened conventional Pap smears that were suspected of being atypical by primary manual screening. The PAPNET archived 128 digital images of the most suspicious cells on the slide. The images were displayed on a video screen for the cytotechnologist to interactively determine whether any of these was suspicious and would require a pathologist to confirm. The intended benefit of the PAPNET was to be as a cell locator to increase the sensitivity of the Pap smear for abnormal cells. The other computer-assisted image-analysis system was the AutoPap 300 QC that was approved for automated screening of all of the conventional Pap smears in a cytology laboratory that were signed out as negative by manual microscopy. The AutoPap 300 QC is no longer sold. It has been replaced by the AutoPap System. The AutoPap System has the extended functionality of automatic primary screening of all low-risk conventional Pap smears and is not limited to selection of the 10% of slides that were previously read as negative by manual microscopy.
The PAPNET device is no longer marketed. The intellectual property of the company was bought by TriPath Imaging, a new company created by merger of NeoPath, the manufacturer of AutoPap, and the AutoCyte Company. Summaries of the safety and effectiveness data that were used to support the FDA approval of the PAPNET and AutoPap devices are listed under TriPath Imaging Company on the FDA website. 4
The third device Sedlacek discussed was Cytyc Corp’s ThinPrep Processor, a device to prepare liquid-based cytology slides to replace conventional Pap smears. A clinician makes conventional Pap smears by collecting cells from the cervix with a spatula and/or brush and then directly smearing the cells adhering to the collection device onto a glass microscope slide. The smeared cells are then either sprayed with a fixative solution or immersed in fixative, usually an alcohol. In contrast, the ThinPrep liquid-based cytology slide is prepared using a collection brush or broom that is immersed directly into an alcohol solution so that the suspended cervical cells are, optimally, rapid-fixed and not allowed to dry out. The ThinPrep filtration process mixes the suspension of cervical epithelial cells into a homogeneous and representative sample that is nearly free of debris, red blood cells, inflammatory cells, and mucus. The summary of the safety and effectiveness data on ThinPrep is on the FDA website under Cytyc Corp. 4 Many postmarket studies have been published on the clinical performance of ThinPrep slides. 5
Sedlacek highlighted the limitations of studies aimed at establishing whether using these three devices or any other diagnostic devices would result in decreased mortality and morbidity from cervical cancer. He warned that using the Eddy model for the cost effectiveness of cytologic screening 6 was not applicable to current medical costs. Sedlacek was looking forward to future studies by the Gynecologic Oncology Group of the American College of Obstetricians and Gynecologists (ACOG), 7 the National Cancer Institute (National Institutes of Health) ASCUS LSIL Triage Study (ALTS), 8 and other studies on new technologies for screening and diagnosis of cervical cancer and its precursors.
Since Sedlacek’s presentation, there have been many excellent publications on the technology assessment of various cervical cytology devices. The following studies use meta-analysis and other evidence-based medicine techniques with excellent descriptions of their methodology and the rationale for the decisions to be drawn for the evidence. The ACOG Committee Opinion report on new Pap test screening techniques is a practice guideline for clinicians that is based on the ACOG’s analysis of the cost-effective use of the new cervical cytology technology. 7 The U.S. Agency for Healthcare Policy and Research [(now Agency for Healthcare Research and Quality (http://www.ahrq.gov/clinic/cervsumm.htm)] commissioned the Evidence-Based Practice Center at Duke University to perform a comprehensive systematic review of the literature on the cost effectiveness of cervical cytology procedures. 9 This study includes recommendations for how to gather and assess the level of evidence in medical literature on cervical cancer screening and diagnosis, and presents a cost-benefit analysis for various proposals for optimal screening of cervical cancer and its precursors. The National Institute for Clinical Excellence of the United Kingdom commissioned its own comprehensive systematic study on liquid-based cytology in cervical screening to assist the National Health Service of England and Wales deciding whether there was a cost benefit to replacing conventional Pap smears with the new technology. 10
1. FDA Center for Devices and Radiologic Health [website]. Available at: http://www.cdrh.fda.gov
. Accessed May 25, 2001.
2. U.S. Food and Drug Administration, Center for Devices and Radiological Health. Determination of safety and effectiveness. [21 CFR 860.7]. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/
cfTopic/topicindex/topindx.cfm?alpha=c. Accessed February 24, 2002.
3. Thornbury JR, Fryback DG. Technology assessment: an American view. Eur J Radiol 1992; 14: 147–156.
4. U.S. Food and Drug Administration, Center for Devices and Radiological Health. Premarket approvals [web site]. FDA-approved devices of TriPath Imaging Inc (PAPNET, AutoPap 300 QC and AutoPap System, and AutoCyte) and Cytyc Corp. Available at: http://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMA/pma.cfm
. Accessed March 7, 2002.
5. Hutchinson ML, Zahniser DJ, Sherman ME, et al. Utility of liquid-based cytology for cervical carcinoma screening: results of a population-based study conducted in a region of Costa Rica with a high incidence of cervical carcinoma. Cancer 1999; 87: 48–55.
6. Eddy D. The frequency of cervical cancer screening: comparison of a mathematical model with empirical data. Cancer 1987; 60: 117–122.
7. ASCUS/LSIL Triage Study (ALTS). Digest page: ALTS cervical cancer screening trial [Cancer Trials, a National Cancer Institute web site]. Available at: http://cancertrials.nci.nih.gov/types/
cervical/alts/index.html. Accessed May 25, 2001.
8. Committee on Gynecologic Practice, American College of Obstetricians and Gynecologists (ACOG). ACOG Committee opinion: new Pap test screening techniques (No. 206; August 1998). Int J Gynaecol Obstet 1998; 63: 312–334.
9. Agency for Healthcare Research and Quality [formerly U.S. Agency for Healthcare Policy and Research (AHCPR)]. Evaluation of Cervical Cytology
. Evidence Report/Technology Assessment No. 5; AHCPR Pub. Nos. 99-E009 (Evidence Report only) and 99-E010 (summary of full evidence report prepared by Duke University, an AHCPR Evidence-Based Practice Center, Durham, NC, under Contract No. 290–97–0014). Available at: 1-800-358-9295 or http://text.nlm.nih.gov/ftrs/dbaccess/cyt
. Accessed May 25, 2001.
10. Payne N, Chilcott J, McGoogan E. Liquid-Based Cytology in Cervical Screening: A Report by the School of Health and Related Research (ScHARR), University of Sheffield, for the National Coordinating Centre for Health Technology Assessment on behalf of the National Institute for Clinical Excellence. January 2000 (revised May 2000;1–99. Available at: http://www
. nice.org.uk. Accessed May 25, 2001.