Skip Navigation LinksHome > October 2007 - Volume 110 - Issue 4 > Interobserver Agreement in the Evaluation of Digitized Cervi...
Obstetrics & Gynecology:
doi: 10.1097/01.AOG.0000281665.63550.8f
Original Articles

Interobserver Agreement in the Evaluation of Digitized Cervical Images

Jeronimo, Jose MD1; Massad, L Stewart MD2; Castle, Philip E. PhD, MPH1; Wacholder, Sholom PhD1; Schiffman, Mark MD, MPH1; for the National Institutes of Health (NIH)–American Society for Colposcopy and Cervical Pathology (ASCCP) Research Group

Free Access
Article Outline
Collapse Box

Author Information

From the 1Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, Maryland; and 2Department of Obstetrics and Gynecology, Washington University School of Medicine, St. Louis, Missouri.

* For the affiliations of the NIH–ASCCP Research Group, see the Appendix.

This is an ancillary study of the National Cancer Institute (NCI)–funded ASCUS–LSIL Triage Study (ALTS). The results do not necessarily reflect the opinions of NCI or the ALTS Investigators.

Corresponding author: Jose Jeronimo, MD, Division of Cancer Epidemiology and Genetics, National Cancer Institute, 6120 Executive Boulevard, Room 5010, Bethesda, MD 20852; e-mail: guibovij@mail.nih.gov.

Financial Disclosure The authors have no potential conflicts of interest to disclose.

Collapse Box

Abstract

OBJECTIVE: To estimate the agreement among multiple expert colposcopists evaluating high-resolution digitized cervigrams taken from patients with a variety of human papillomavirus (HPV) infection states and previous cervigram interpretations.

METHODS: Twenty expert colposcopists evaluated 939 digitized images of the uterine cervix obtained after the application of 5% acetic acid during the ASCUS-LSIL Triage Study. Twenty images selected to represent a broad range were graded by all the colposcopists. The remaining 919 pictures were distributed by stratified random sampling, such that each image was evaluated by two colposcopists, and each expert evaluated 112 images with similar distributions of cervigram diagnoses and HPV DNA test results. We evaluated interrater agreement among the pairs of colposcopists and confirmed the conclusions using the 20 images they all graded.

RESULTS: Pairs of colposcopists agreed on the diagnosis for only 56.8% of images. Similar agreement was seen regarding number of visible lesions (of low-grade or greater). This variability in ratings remained when the images were stratified by final histologic diagnosis or HPV status. The results were confirmed by the presence of large variability in ratings (ranging in some cases from normal to cancer) for the 20 images graded by all colposcopists.

CONCLUSION: Colposcopic diagnosis using static images is poorly reproducible and might reflect similar problems in clinical practice. Researchers should question the use of colposcopic images as a reference standard for teaching and evaluating the presence or severity of disease.

LEVEL OF EVIDENCE: II

Cervical cancer is preceded by a long-lasting precancerous phase that starts when women are infected with one or more of the carcinogenic types of the human papillomavirus (HPV).1,2 More than 90% of the HPV-infected women will clear their infections within 2 years. Conversely, only a small percentage of them will develop persistent infection, which is considered the truly high-risk state for developing precancer and cancer.

Cervical cancer rates have fallen dramatically over the past four decades. The success of cervical cancer prevention is mainly due to the easy access of cervical epithelium for direct evaluation of microscopic and visual signs of HPV infection before development of cancer. Specifically, when abnormalities are detected during screening, magnified visual evaluation of the cervix (colposcopy) plays a critical role in the diagnosis of precancer and cancer. If an acetowhite area suggesting possible cervical intraepithelial neoplasia (CIN) is observed, characteristics of the lesions, such as border, intensity of color and texture, and vascular patterns, are used to predict severity of disease.3 The areas of the lesion(s) with the most atypical changes or highest lesion score are targeted for biopsy. The diagnosis provided by colposcopy and colposcopically directed biopsy guide the treatment of the patient, and the colposcopic procedure (including biopsy) has been considered the reference standard of health and disease in clinical practice and research.

We are currently evaluating rigorously how to improve the teaching and practice of colposcopy because of the recent evidence that colposcopy is only about 70% sensitive for detection of precancerous lesions.4,5 Static pictures of the uterine cervix are widely used for training and testing colposcopists. Cervigrams are 35 mm Kodachrome (Kodak, Rochester, NY) photographs taken with a fixed focal length camera with a 100 mm macro lens. Evaluators obtain a magnified view of the cervix when those images are projected on a screen. During the ASCUS-LSIL Triage Study (ALTS), cervigrams were taken at enrollment and during the 2 years of follow-up of more than 5,000 patients.

Previous reports have shown a lack of agreement among colposcopists, but some of the studies were based on low-resolution visual material.6–8 This study was designed to estimate the agreement among multiple expert colposcopists evaluating high-resolution digitized cervigrams taken from patients with a variety of HPV infection states (single type, multiple types, carcinogenic types, noncarcinogenic types, etc), and previous cervigram interpretations (equivalent to normal, atypical, low-grade, high-grade, and suspect for invasive cancer).

Back to Top | Article Outline

MATERIALS AND METHODS

Details of ALTS and characteristics of its study population have been previously described.9 Briefly, a total of 5,060 women were enrolled in the study between November 1996 and December 1998 because they were referred with a cytologic report of atypical squamous cell of unknown significance (ASC-US) or low-grade squamous intraepithelial lesion (LSIL). Patients were randomly assigned to three management strategies (immediate colposcopy, triage based on HPV status and thin-layer cytology results, or triage based on cytology results only). The study took place in four clinical settings: Magee-Women’s Hospital of the University of Pittsburgh Medical Center Health System (Pittsburgh, PA), the University of Oklahoma (Oklahoma City, OK), the University of Alabama (Birmingham, AL), and the University of Washington (Seattle, WA). It was approved by local institutional review boards and in accordance with the U.S. Department of Health and Human Services norms. Each woman signed a written informed consent document at enrollment allowing image capture and analysis. Samples for liquid-based cytology and HPV testing, and complete questionnaire data were collected.

A sample for cytology was collected with a broom-type sampler (Papette Cervical Cell Collector, Wallach Surgical Devices, Incorporated, Orange, CT) and transferred to a transport medium (PreservCyt, Cytyc Corporation, Marlborough, MA). Another sample was collected with a fiber-tipped (Dacron, E.I. du Pont de Nemours and Company, Wilmington, DE) swab, put into Specimen Transport Medium (Digene Corporation, Gaithersburg. MD). Line Blot Assay (Roche Molecular Systems, Pleasanton, CA) was used for detection of 38 individual carcinogenic and noncarcinogenic HPV genotypes.10 Human papillomavirus results were categorized hierarchically according to cancer risk (HPV16 else carcinogenic HPV excluding HPV16 else noncarcinogenic HPV else HPV negative). Once all the cytologic samples were collected, the cervix was washed with 5% acetic acid for 1 minute and two cervigrams were taken.

By stratified random sampling, we selected 1,000 women evaluated at enrollment into ALTS based on their HPV status and initial cervigram diagnosis. Cervigrams from 21 women were lost, and 40 were of poor quality and inadequate for clinical evaluation. The final sample for this study consisted of 939 cervigrams from the same number of patients at the time of their enrollment into ALTS. The group, as designed, had the following variety of characteristics:

1. No HPV infection and normal cervigrams (n=98; 10.4% of the total sample of 939)

2. No HPV infection detected and cervigram considered as atypical or low-grade lesion (n=114; 12.1% of the total sample)

3. No HPV infection detected and cervigram with diagnosis of high-grade lesion or cancer (n=10; 1.1% of the total sample)

4. Infected with noncarcinogenic HPV and normal cervigram (n=36; 3.8% of the total sample)

5. Infected with noncarcinogenic HPV and cervigram considered as atypical or low-grade lesion (n=69; 7.3% of the total sample)

6. Infected with noncarcinogenic HPV and cervigram with diagnosis of high-grade lesion or cancer (n=10; 1.1% of the total sample)

7. Infected with carcinogenic HPV including HPV16 and normal cervigram (n=123; 13.1% of the total sample)

8. Infected with carcinogenic HPV including HPV16 and cervigram considered as atypical or low-grade lesion (n=387; 41.2% of the total sample)

9. Infected with carcinogenic HPV including HPV16 and cervigram with diagnosis of high-grade lesion or cancer (n=92; 9.8% of the total sample)

Within the group of 939, cervigrams from 20 women were selected, again by stratified random sampling, for evaluation by all the evaluators in the study. This subsample was chosen to represent clear contrasts: we chose women with CIN 1 or less who either had no HPV (n=9) or carcinogenic HPV (n=7), and women with CIN 3 (n=4) all of whom had carcinogenic HPV. We avoided the ambiguity of cases of CIN 2 and noncarcinogenic HPV, and the very few cases of CIN 3 without carcinogenic HPV.

The remaining 919 pairs of cervigrams were randomly assigned in such a way that each picture had at least two evaluators; each evaluator had a set of 112 cervigrams with similar proportions of cases from each one of the groups of patients; and each evaluator shared five cases with every one of the other colposcopists (in addition to the 20 images shared by all colposcopists). Diagnoses were rendered and, even though a formal scoring was not required, evaluators were asked to rate each cervix for color, margin, acetowhitening, and vascularity.

All patients was given a final study diagnosis during the ALTS study (CIN 3, CIN 2, or CIN 1 or less) that was based on the worst histology at enrollment and during the 2 years of follow-up. The study diagnoses were assigned by a quality control group of pathologists.

Optimal features for digitization and compression of 35 mm cervigrams have been previously described.11 Software developed by the National Institutes of Health, called the Boundary Marking Tool,12 was used to access and evaluate the images through the internet. Evaluators were masked to any clinical data of the patients such as histologic diagnosis, cytology or HPV status. The evaluators were 20 expert colposcopists (12 general gynecologists and 8 gynecologist–oncologists) with more than 10 years of experience in the field. Before starting evaluation of the cases, all the colposcopists completed pilot cases. Even though computers and monitors used by the evaluators were not standardized, a detailed manual for using the software and protocol for rating and marking images was provided to all.

During the evaluation, performed between January and April 2006, colposcopists were required to draw a boundary around any acetowhite lesion and to score visual characteristics such as punctation, mosaicism, borders, and color of the lesions using an index modified from the one described by Reid and Scalzi.3 Additionally, they provided a colposcopic diagnosis for each lesion and an overall diagnosis for the entire case based on the area with the most abnormal-appearing lesion. The following diagnoses could be selected: normal, cervicitis, metaplasia, condyloma, low-grade lesion (CIN 1), or high-grade lesion (CIN 2–3 or cancer).

We compared the results from the individual evaluators, with a focus on interrater reproducibility. Because some images are far easier to evaluate than others, statistical tests that require independence of observations are not appropriate. Therefore, we relied on simple descriptions of frequency of evaluations for the subset of 20 was focused on the presence or absence of discernible lesions, number of lesions drawn and overall case diagnosis. Specifically, we tabulated the frequency of diagnoses for each of the 20 images rated by all colposcopists to show the wide range of interpretations. Similarly, we tabulated the frequency of all diagnoses given for each of the colposcopists for all 112 of their ratings, again to show the broad range despite the balanced severity of the random study sets given to each colposcopist.

We used ordered logistic regression to model the evaluations of the individual colposcopists. Ordered logistic regression models13 were applied to categorical outcomes by ranking outcomes low to high with no assumption of the “distance” between categories. The models were used to predict probabilities that each colposcopist would assign a specific diagnosis to each cervigram, adjusting for other covariates, including patient age (younger than 21, 21–24, 25–29, and 30 or older); HPV status (negative or noncarcinogenic HPV, carcinogenic HPV excluding HPV16, and HPV16); and final histologic diagnosis (less than CIN 2, CIN 2, and CIN 3). The models also predicted the likelihood of each outcome within the categories of age and HPV status. To test for linear trends on a log scale, we treated age categories and HPV status as continuous in the multivariable models. For this analysis we ignored the possible autocorrelation between interpretations of the same image by two reviewers in each pair. As a result, the overall patterns of visual diagnoses by individual raters will likely seem more similar than would otherwise be observed.

Finally, we tabulated for each of the reviewer pairs their ratings for the 919 images (apart from the 20 rated by all), with regard to image severity and numbers of lesions (low grade or worse). These tabulations permitted simple examination of amount of agreement between random pairs of experts regarding core aspects of cervical evaluation (severity of impression and presence and number of lesions that might require biopsy).

Back to Top | Article Outline

RESULTS

Digitized pictures of the uterine cervix of 1,000 women evaluated at enrollment were considered for this study. The mean age was 26.2 years old (standard deviation 7.8 years) with a median of 24 (range 18–73) years. The community-based cytology that prompted referral to ALTS was ASC-US in 577 (61.5%) and LSIL in 362 (38.8%). Of the 939 women included in this analysis, 222 women (23.6%) were negative for HPV, 115 (12.3%) were positive only for noncarcinogenic types of HPV, and 602 (64.1%) were positive for at least one carcinogenic type of HPV. Regarding the final ALTS diagnoses, 684 cases (72.8%) had CIN 1 or less, 83 (8.8%) had CIN 2, and 172 (18.3%) had CIN 3 or worse, which included three cases of cancer. There was no meaningful difference in the HPV status and the diagnosis of the cervigram taken at enrollment of these 61 cases who were excluded from this analysis and the 939 who were included in this analysis.

Figure 1 shows the visual diagnosis provided by all the evaluators for 20 randomly selected cases. The severity of colposcopic impression tended to increase with worse underlying pathology; however, substantial variability existed, eg, in the nine patients with negative HPV tests and final diagnoses of CIN 1 or less. Of these nine patients, four were considered as normal by almost all the evaluators, four others had diagnoses ranging from normal to high-grade lesion, and for one patient most of the evaluators agreed on the presence of high-grade lesion even though that patient had a negative HPV test and no evidence of precancer during 2 years of follow-up. Evaluator agreement was greatest for the four women with final diagnosis of CIN 3 and positive test results for carcinogenic types of HPV.

Fig. 1
Fig. 1
Image Tools

We had produced comparable groups of patients by random sampling for each evaluator; therefore, we could compare the frequency of diagnoses provided by each colposcopist for his or her entire set of cases. Figure 2 shows that the frequency of diagnoses varied widely among evaluators, with the greatest variability in the frequency of normal or metaplasia diagnoses that ranged from 11.6% for evaluator 1 to 60.7% for evaluator 20 (range 49.1%). The variability was similar even when restricting to cytologic diagnosis: the range for high-grade (29.8%) was slightly smaller than for low-grade diagnosis (43.8%). Because a colposcopic diagnosis of metaplasia could be related more closely to that of low-grade lesion (LGL) than to normality, we regrouped patients as colposcopically normal, metaplasia/LGL and high-grade lesion or cancer (HGL+); nonetheless, we observed the same heterogeneity among the observers.

Fig. 2
Fig. 2
Image Tools

In an attempt to explore factors affecting the colposcopic diagnoses of the evaluators, we examined the differences between raters using a multivariable analysis (Table 1) that adjusted for HPV type, final histologic diagnosis, and patient age. It was important to adjust for these possibly confounding factors in case of residual imperfections in our attempts by stratified random sampling to produce sets of images that were comparable. Women infected with HPV16 were slightly more likely, on average, to be diagnosed as LGL or HGL (60%) than women with non-HPV16 carcinogenic types (53%) or those infected with noncarcinogenic types or uninfected (51%). The frequency of visual diagnosis was very similar between women aged younger than 21 and women aged between 21 and 29 years; but women aged older than 30 years old were less likely to be diagnosed with HGL.

Table 1
Table 1
Image Tools

Table 1 also shows the frequencies of diagnoses and the ratio of HGL to normal diagnoses per evaluator, adjusted for HPV, histology, and age. Despite adjustment, the visual diagnoses of the evaluators varied widely. It was possible to observe the wider variability in the diagnosis of normal or cervicitis or metaplasia, the frequency of which ranged from 25% (Evaluator 1) to 65% (Evaluator 20).

Table 2 shows the agreement of the diagnosis of pairs of colposcopists in the 919 women evaluated only by two experts. In 56.8% of cases the evaluators had complete agreement in the diagnosis. In 37.8% of cases, one of the evaluators diagnosed LGL and the other evaluator considered it normal or HGL. In the remaining 5.3% of women there was absolute disagreement because one evaluator diagnosed as normal a case considered as HSL by another expert. It is of note that there was agreement on HGL for only a minority of lesions classified by at least one reviewer as HGL {97/[97+139+46]=34.4%}.

Table 2
Table 2
Image Tools

We repeated the evaluation in Table 2 in subgroups of patients according to their HPV status and histologic outcome. As a striking observation, we observed that even though the percentage of overall agreement was similar within HPV strata to the agreement in the total sample, the percentage of agreement on high-grade lesions increased from 4.9% in the HPV-negative or noncarcinogenic HPV-positive women to 10.7% in the carcinogenic HPV-positive (excluding HPV16) women and to 22.3% in the HPV16-positive women. Similarly, the percentage of agreement on visual diagnosis of high-grade lesions increased from 7.0% in women with final histology of less than CIN 2 to 22.1% in women with final histologic diagnosis of CIN 2 or worse.

In Table 3 we tabulated the agreement in the number of lesions drawn by pair of evaluators. With the objective of simplifying the evaluation of the results, we grouped the number of lesions in three categories (0, 1, and 2 or more). The evaluators agreed on the number of lesions in 53.4% of the cases, but the level of agreement was primarily because 25.8% were cases in which no visible lesions were observed by any reviewer; among the images in which one reviewer reported at least one lesion, the agreement was only 37.2%, but was better in women with HPV16 (43.7%) than women with negative or noncarcinogenic HPV (34.5%). We also found a better agreement in patients with final histologic diagnosis less than CIN 2 (56.4%) than in CIN 2+ cases (45.9%), probably again because the percentage of women without a visible lesion was higher in the first group (32.4% compared with 9.0%).

Table 3
Table 3
Image Tools
Back to Top | Article Outline

DISCUSSION

The ASCUS–LSIL Triage Study has provided several examples in gynecology and cervical cancer prevention of the interobserver variability that is inevitable in all human evaluation. Stoler et al14 used cytologic and histologic specimens collected during ALTS and reported a lack of reproducibility for cytology, punch-biopsy histology and loop electrosurgical excision procedure histologic evaluation (κ=0.46, 0.46, and 0.49, respectively). Similarly, Ferris et al7 reported the results of interobserver agreement among three expert colposcopists using digital pictures taken during the colposcopic evaluation (DenVu, Tucson, AZ) in ALTS. They found fair to poor agreement for colposcopic diagnosis, Reid Index scores, and size of the lesion.

Our study extended the analysis to multiple expert colposcopists using high-resolution digitized cervigrams and a novel Web-based software to collect their evaluations. The evaluators for this study were highly trained and experienced colposcopists, even though in a previous report5 we found that the sensitivity of the overall colposcopic procedure, including biopsies, does not necessarily differ by type of medical training.

We found a lack of agreement in estimated number of lesions and diagnoses provided by the expert colposcopists. There are several factors that might influence the frequency of visual diagnosis provided by each evaluator. We found that HPV status and age of the patient are independent factors influencing the outcome of the colposcopic evaluation. In particular, women with HPV16 are more likely than women with other oncogenic types to have a high-grade visual lesion and resulted in better interobserver agreement. This genotype causes the highest risk for chronic infection and development of precancer.15 Moreover, in a previous report16 our research group showed that, taking severity of the underlying pathology into account, independent of the underlying grade of CIN, HPV16 is more likely to produce a clinically identifiable lesion than other HPV types.

An even higher risk for CIN 3 is expected in the presence of a cytologic result of high-grade squamous intraepithelial lesions.17 Also, colposcopists were more likely to diagnose high-grade lesions in women aged younger than 30 years. We expected this age-related result because in ALTS the mean and median age of patients with histologic diagnosis of CIN 3 was approximately 25 years. After adjusting for possible confounding factors that persisted despite our random sampling of images, we found strong differences in ratings of digitized cervical images by expert evaluators. The agreement among the whole group of evaluators was poor.

Our study was limited to static images and cannot necessarily be generalized to a colposcopic examination. The results indicate strongly that we must be cautious when using even high-quality static images to teach and test colposcopists. The results, by extension, support growing evidence raising concern about suboptimal agreement among highly trained and experienced colposcopists.7,18 If the colposcopic impression has only fair reproducibility, then it cannot serve as a good reference standard of health or disease.

Our results suggest the need to study the potential benefit of taking more than one colposcopically directed biopsy, or even random biopsies when there is no clear lesion, to improve the sensitivity of colposcopy and reproducibility of the colposcopic biopsy-derived diagnosis regardless of the diagnostic impression. Optimal biopsy strategies should be established in rigorous trials. In ongoing studies, we are moving to assessment of choice of biopsy placement, which is perhaps the critical function of the colposcopic examination.

Back to Top | Article Outline

REFERENCES

1. Walboomers JM, Jacobs MV, Manos MM, Bosch FX, Kummer JA, Shah KV, et al. Human papillomavirus is a necessary cause of invasive cervical cancer worldwide. J Pathol 1999;189:12–9.

2. Bosch FX, Lorincz A, Munoz N, Meijer CJ, Shah KV. The causal relation between human papillomavirus and cervical cancer. J Clin Pathol 2002;55:244–65.

3. Reid R, Scalzi P. Genital warts and cervical cancer. VII. An improved colposcopic index for differentiating benign papillomaviral infections from high-grade cervical intraepithelial neoplasia. Am J Obstet Gynecol 1985;153:611–8.

4. Guido R, Schiffman M, Solomon D, Burke L, ASCUS LSIL Triage Study (ALTS) Group. Postcolposcopy management strategies for women referred with low-grade squamous intraepithelial lesions or human papillomavirus DNA-positive atypical squamous cells of undetermined significance: a two-year prospective study. Am J Obstet Gynecol 2003;188:1401–5.

5. Gage JC, Hanson VW, Abbey K, Dippery S, Gardner S, Kubota J, et al. Number of cervical biopsies and sensitivity of colposcopy. Obstet Gynecol 2006;108:264–72.

6. Etherington IJ, Luesley DM, Shafi MI, Dunn J, Hiller L, Jordan JA. Observer variability among colposcopists from the West Midlands region. Br J Obstet Gynaecol 1997;104:1380–4.

7. Ferris DG, Litaker M. Interobserver agreement for colposcopy quality control using digitized colposcopic images during the ALTS trial. J Low Genit Tract Dis 2005;9:29–35.

8. Baum ME, Rader JS, Gibb RK, McAlister RP, Powell MA, Mutch DG, et al. Colposcopic accuracy of obstetrics and gynecology residents. Gynecol Oncol 2006;103:966–70.

9. Schiffman M, Adrianza ME. ASCUS-LSIL Triage Study. Design, methods and characteristics of trial participants. Acta Cytol 2000;44:726–42.

10. Peyton CL, Gravitt PE, Hunt WC, Hundley RS, Zhao M, Apple RJ, et al. Determinants of genital human papillomavirus detection in a US population. J Infect Dis 2001;183:1554–64.

11. Jeronimo J, Long R, Neve L, Ferris D, Noller K, Spitzer M, et al. Preparing digitized cervigrams for colposcopy research and education: determination of optimal resolution and compression parameters. J Low Genit Tract Dis 2006;10:39–44.

12. Jeronimo J, Long LR, Neve L, Michael B, Antani S, Schiffman M. Digital tools for collecting data from cervigrams for research and training in colposcopy. J Low Genit Tract Dis 2006;10:16–25.

13. Long JS. Regression Models for categorical and limited dependent variables. London (UK): SAGE Publications; 1997.

14. Stoler MH, Schiffman M, Atypical Squamous Cells of Undetermined Significance-Low-grade Squamous Intraepithelial Lesion Triage Study (ALTS) Group. Interobserver reproducibility of cervical cytologic and histologic interpretations: realistic estimates from the ASCUS-LSIL Triage Study. JAMA 2001;285:1500–5.

15. Khan MJ, Castle PE, Lorincz AT, Wacholder S, Sherman M, Scott DR, et al. The elevated 10-year risk of cervical precancer and cancer in women with human papillomavirus (HPV) type 16 or 18 and the possible utility of type-specific HPV testing in clinical practice. J Natl Cancer Inst 2005;97:1072–9.

16. Jeronimo J, Massad LS, Schiffman M, National Institutes of Health/American Society for Colposcopy and Cervical Pathology (NIH/ASCCP) Research Group. Visual appearance of the uterine cervix: correlation with human papillomavirus detection and type. Am J Obstet Gynecol 2007;197:47.e1–8.

17. Wang SS, Walker JL, Schiffman M, Solomon D. Evaluating the risk of cervical precancer with a combination of cytologic, virologic, and visual methods [published erratum appears in Cancer Epidemiol Biomarkers Prev 2006;15:187]. Cancer Epidemiol Biomarkers Prev 2005;14:2665–8.

18. Pretorius RG, Zhang WH, Belinson JL, Huang MN, Wu LY, Zhang X, et al. Colposcopically directed biopsy, random cervical biopsy, and endocervical curettage in the diagnosis of cervical intraepithelial neoplasia II or worse. Am J Obstet Gynecol 2004;191:430–4.

Back to Top | Article Outline
APPENDIX

Affiliations of the National Institutes of Health (NIH)–American Society for Colposcopy and Cervical Pathology (ASCCP) Research Group:

Lori Boardman, Obstetrician Gynecologist, Department of Obstetrics and Gynecology, Women and Infants’ Hospital, Providence, RI; Peter Cartwright, Obstetrician Gynecologist, Department of Obstetrics and Gynecology Duke University, NC; Philip Castle, Investigator, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD; Charles Dunton, Gynecologist Oncologist, Division of Gynecologic Oncology, Lankenau Hospital, Philadelphia, PA; Julia Gage, Investigator, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD; Richard Guido, Obstetrician Gynecologist, Magee-Women’s Hospital of the University of Pittsburgh Health Care System, Pittsburgh, PA; Fernando Guijon, Obstetrician Gynecologist, Department of Obstetrics, Gynecology and Reproductive Sciences, University of Manitoba, Canada; Thomas Herzog, Gynecologist Oncologist; Columbia University, New York, NY; Warner Huh, Gynecologist Oncologist, University of Alabama at Birmingham, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, Birmingham, AL; Jose Jeronimo, Gynecologist Oncologist, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD; Abner Korn, Obstetrician Gynecologist, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, CA; Edward Kost, Gynecologist Oncologist, Division of Gynecologic Oncology, Brooke Army Medical Center, Fort Sam Houston, TX; Ramey D. Littell, Gynecologist Oncologist, Kaiser Permanente, San Francisco Medical Center, CA; Rodney Long, Engineer visual data management, Communications Engineering Branch, National Library of Medicine, Bethesda, MD; Stewart Massad, Gynecologist Oncologist, Department of Obstetrics and Gynecology, Washington University School of Medicine, St. Louis, MO; Jorge Morales, Obstetrician Gynecologist, Proyecto Epidemiologico Guanacaste, Costa Rica; Leif Neve, visual data management, Communications Engineering Branch, National Library of Medicine, Bethesda, MD; Dennis O’Connor, Gynecologic Pathologist, CPA Laboratory, Louisville, KY; Janet S. Rader, Gynecologist Oncologist, Department of Obstetrics and Gynecology, Division of Gynecologic Oncology, Washington University School of Medicine, St. Louis, MO; George Sawaya, Obstetrician Gynecologist, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, CA; Mark Schiffman, Division of Cancer Epidemiology and Genetics, National Cancer Institute, Bethesda, MD; Mario Sideri, Gynecologist Oncologist, Division of Gynecology, European Institute of Oncology, Milan, Italy; Karen Smith-McCune, Obstetrician Gynecologist, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, CA; Mark Spitzer, Obstetrician Gynecologist, Department of Obstetrics and Gynecology, Brookdale University Hospital, Brooklyn, NY; Alan Waxman, Obstetrician Gynecologist, Departments of Obstetrics and Gynecology, University of New Mexico Health Sciences Center, Albuquerque, NM; Claudia Werner, Obstetrician Gynecologist, Department of Obstetrics and Gynecology, University of Texas Southwestern Medical Center at Dallas, Dallas, TX. Cited Here...

Figure. No caption a...
Image Tools

Cited By:

This article has been cited 13 time(s).

Gynecologie Obstetrique & Fertilite
Proposal of a new methodology: Tridimensional colposcopy (3D) of the lower genital tract
Coppolillo, E; Tauscher, P; Perazzi, B; Boulanger, JC
Gynecologie Obstetrique & Fertilite, 41(): 471-474.
10.1016/j.gyobfe.2013.05.004
CrossRef
Obstetrics and Gynecology Clinics of North America
Cervical Cancer Prevention New Guidelines in the United States and New Opportunities for Low- and Middle-Income Countries
Waxman, AG
Obstetrics and Gynecology Clinics of North America, 40(2): 251-+.
10.1016/j.ogc.2013.03.006
CrossRef
Forensic Science Medicine and Pathology
Colposcopic photography of genital injury following sexual intercourse in adults
Astrup, BS; Lauritsen, J; Thomsen, JL; Ravn, P
Forensic Science Medicine and Pathology, 9(1): 24-30.
10.1007/s12024-012-9394-2
CrossRef
Acta Obstetricia Et Gynecologica Scandinavica
Three-dimensional digital colposcopy
Coppolillo, EF; Eliseth, MC; Vega, HMDR; Perazzi, BE
Acta Obstetricia Et Gynecologica Scandinavica, 88(1): 120-121.
10.1080/00016340802534802
CrossRef
Cancer
Human papillomavirus infection and the primary and secondary prevention of cervical cancer
Lowy, DR; Solomon, D; Hildesheim, A; Schiller, JT; Schiffman, M
Cancer, 113(7): 1980-1993.
10.1002/cncr.23704
CrossRef
Journal of Biomedical Optics
Using acetowhite opacity index for detecting cervical intraepithelial neoplasia
Li, WJ; Venkataraman, S; Gustafsson, U; Oyama, JC; Ferris, DG; Lieberman, RW
Journal of Biomedical Optics, 14(1): -.
ARTN 014020
CrossRef
Obstetrics and Gynecology Clinics of North America
A Tissue Basis for Colposcopic Findings
O'Connor, DM
Obstetrics and Gynecology Clinics of North America, 35(4): 565-+.
10.1016/j.ogc.2008.09.009
CrossRef
New England Journal of Medicine
Human papillomavirus DNA versus papanicolaou screening tests for cervical cancer - Reply
Franco, EL; Mayrand, MH; Ratnam, S
New England Journal of Medicine, 358(6): 643.

International Journal of Cancer
How to evaluate emerging technologies in cervical cancer screening?
Arbyn, M; Ronco, G; Cuzick, J; Wentzensen, N; Castle, PE
International Journal of Cancer, 125(): 2489-2496.
10.1002/ijc.24774
CrossRef
Medical Imaging 2008: Image Processing, Pts 1-3
Computerized image analysis for acetic acid induced intraepithelial lesions - art. no. 69143A
Li, WJ; Ferris, DG; Lieberman, RW
Medical Imaging 2008: Image Processing, Pts 1-3, 6914(): A9143-A9143.

Gynecologic Oncology
Accuracy of cervical specimens obtained for biomarker studies in women with CIN3
Wentzensen, N; Zuna, RE; Sherman, ME; Gold, MA; Schiffman, M; Dunn, ST; Jeronimo, J; Zhang, R; Walker, J; Wang, SS
Gynecologic Oncology, 115(3): 493-496.
10.1016/j.ygyno.2009.09.001
CrossRef
Obstetrics & Gynecology
More Questions About the Accuracy of Colposcopy: What Does This Mean for Cervical Cancer Prevention?
Cox, JT
Obstetrics & Gynecology, 111(6): 1266-1267.
10.1097/AOG.0b013e3181792420
PDF (139) | CrossRef
Obstetrics & Gynecology
Interobserver Agreement in the Assessment of Components of Colposcopic Grading
Massad, LS; Jeronimo, J; Schiffman, M; for the National Institutes of Health/American Society for Colposcopy and Cervical Pathology (NIH/ASCCP) Research Group,
Obstetrics & Gynecology, 111(6): 1279-1284.
10.1097/AOG.0b013e31816baed1
PDF (738) | CrossRef
Back to Top | Article Outline

© 2007 The American College of Obstetricians and Gynecologists

Login

Article Tools

Images

Share