Background: Errors in the classification of male circumcision status could bias studies linking infection to lack of circumcision.
Goal: To determine the frequency and factors associated with the reproducibility of reporting circumcision status.
Study Design: Secondary analysis of data using logistic regression modeling from a multicenter randomized controlled trial was performed.
Results: At follow-up assessment, 15.6% of clinician reports on circumcision status disagreed with baseline reports. Disagreement was more common if both clinicians were women than if both were men (odds ratio [OR], 2.8; 95% CI, 1.9–4.1). As compared with whites reported as circumcised (4%, 19/532 visits), the highest disagreement involved uncircumcised Hispanic (OR, 3.3; 95% CI, 1.7–6.3), white (OR, 12.2; 95% CI, 5.8–25.6), or black (OR, 17.1; 95% CI, 10.4–27.9) men.
Conclusions: This is one study among a small number of studies examining the reproducibility of clinician-reported circumcision status by comparing multiple clinical examinations of the same patient. The magnitude of the misclassification discovered could bias results and indicates the need for greater accuracy in reporting circumcision status in future studies.
From *Kaiser Permanente, Atlanta, Georgia; †Centers for Disease Control and Prevention, National Center for HIV, STD, and TB Prevention, Division of HIV/AIDS Prevention, Atlanta, Georgia; ‡Project RESPECT Study Group, San Francisco Health Department, San Francisco, California; §Project RESPECT Study Group, Baltimore City Health Department and Johns Hopkins University, Baltimore, Maryland; ‖‖Project RESPECT Study Group, Denver Public Health Department, Denver, Colorado; ¶Project RESPECT Study Group, Long Beach Health Department and California State University, Long Beach, California; and #Project RESPECT Study Group, New Jersey Health Department and Newark STD Clinic, Newark, New Jersey
The authors thank the Project RESPECT team for collecting the data used in this study, especially those who went out of their way to track down clinician data for this analysis. They particularly thank Mr. Daniel Newman and Dr. Julie Maher for their assistance with the data analysis. The Project RESPECT study group includes Baltimore: Carolyn Erwin-Johnson, MA (Intervention Coordinator [IC]), Andrew L. Lentz, MPA (Project Manager [PM]), Mary A. Staat, MD, MPH (coprincipal investigator [co-PI]), Jonathan M. Zenilman, MD (principal investigator [PI]); Denver: John M. Douglas (PI), Tamara Hoxworth, PhD (IC), Ken Miller, MPH (PM), William McGill, PhD; Long Beach: Ruth Bundy, PhD (co-PI), Laura A. Hoyt, MPA (PM), C. Kevin Malotte, DrPH (IC), Fen Rhodes, PhD (PI); Newark: Michael Iatesta, MA (PM), Eileen Napolitano (co-PI), Judy Rogers, MS (IC), Ken Spitalny, MD (PI); San Francisco: Gail A. Bolan, MD (PI), Coleen LeDrew, Kimberly A. J. Coleman (IC), Luna Hananel, MSW (IC), Charlotte K. Kent, MPH (PM); Nova, Inc, Bethesda: Robert Francis, PhD (PI), Christopher Gordon, Nancy Rosenshine, MA (PI and IC), Carmita Signes; CDC: Sevgi Aral, PhD, Robert H Byers, PhD, Beth Dillon, MSW, Martin Fishbein, PhD, Sandra Graziano, PhD, Russ Havlak, Mary L. Kamb, MD, MPH, William Killean, James Newhall, PhD, Daniel Newman, MS, Thomas A. Peterman, MD, MSc, Karen L. Willis, RN.
Reprint requests: National Center for HIV, STD, and TB Prevention, Office of Communications, Centers for Disease Control and Prevention, 1600 Clifton Road, Mail Stop E-06, Atlanta, Georgia 30333.
Received for publication July 28, 2000, revised October 30, 2000, and accepted November 16, 2000.
Approximately one sixth of circumcision status follow-up reports were inconsistent with baseline reports. Errors were more likely for female clinicians, African American patients, and men reported as uncircumcised at baseline.
SEVERAL STUDIES HAVE SHOWN that uncircumcised men have a higher risk than circumcised men for acquiring HIV and other sexually transmitted diseases (STDs) such as chancroid, chlamydia, gonorrhea, syphilis, genital herpes, and genital warts. 1–14 Other studies, however, have found no association between circumcision and the acquisition of STDs or HIV. 15–18 Misclassification of circumcision status would affect the observed relation with STD or HIV infection.
The possibility of misclassified circumcision status, especially in self-report data, has been studied. Research has shown that circumcision status determined by physical examination is more accurate than patients’ statements about their own circumcision status. 10–14,19–22 Few studies, however, have examined the reproducibility of the clinician’s report on patient circumcision status. Most studies of circumcision status have been cross-sectional investigations, with only one measure of circumcision status. The authors had a unique opportunity to examine the reproducibility of clinician reports on circumcision status in a large longitudinal study during which patients were examined multiple times by different clinicians.
Patients attending public STD clinics between July 1993 and September 1996 were invited to enroll in a randomized controlled trial (Project RESPECT) conducted at five study sites: Baltimore, Maryland; Denver, Colorado; Long Beach, California; Newark, New Jersey; and San Francisco, California. 23 The patients were examined by a clinician (physician’s assistant, nurse practitioner, registered nurse, or medical doctor) at baseline, treated for STDs, and reexamined during visits scheduled 6 and 12 months later. The patients also were reexamined if they returned to the clinic with STD symptoms, if they wanted a checkup, or if an infected partner referred them to the clinic. As an item on the medical record form, circumcision status was assessed by study clinicians at each examination. Examinations performed during follow-up visits were as thorough as those performed at baseline.
To identify factors associated with reproducibility of report, circumcision status reported for a man at baseline was compared with circumcision status reported at each follow-up visit to see whether they agreed. Follow-up visits in which the clinician report was consistent with the baseline report were compared with visits for which reported circumcision status differed. Men who were scheduled and who returned for at least one follow-up visit were included in the analysis.
The following factors were considered: study site, patient characteristics (circumcision status noted at baseline, age, race/ethnicity, symptoms or, diagnosis of a STD [gonorrhea, chlamydia or syphilis]); clinician characteristics (gender, race/ethnicity, academic training); and whether the same clinician saw the patient at the baseline and follow-up visits. Although designated clinicians were assigned to work on the study and trained to perform examinations according to study protocol, some clinicians who substituted for study clinicians had identification numbers that could not be matched to a name. These were considered nonstudy clinicians. The study protocol did not include specific training for the clinicians in classifying circumcision status according to a standard procedure. The usual practice was for the clinicians to handle and directly observe the penis.
To determine whether some individuals were more likely to be reported consistently than others, each individual’s reported circumcision status at each visit was compared with his reported circumcision status at every other visit. Using the overall observed chance of disagreement (from the original analysis, 0.156), the estimated likelihood of repeated disagreement among patients with four or more examinations was calculated.
The probability of agreement was analyzed using generalized estimating equations to model logistic regression within SAS software version 6.12 (SAS Institute, Cary, NC), which produced estimated odds ratios (OR) and 95% CI. This method allowed for multiple observations per individual and modeled the effect of patient- and clinician-specific variables. Circumcision status reported at baseline was used as a proxy for true circumcision status. The final model determined by using the likelihood ratio test to eliminate logical groupings of categorical variables followed by backward elimination of individual variables. The study site was omitted from the modeling because it was highly correlated with the clinician variables.
Of the 3269 men enrolled in Project RESPECT, the following were omitted from the study: 812 men who were not scheduled (by design) for follow-up visits, 564 men who were scheduled but did not return for any follow-up visits, 10 men (40 visits) who were seen by physicians, and 33 men whose circumcision status was not recorded at any visit. This left 1850 baseline and 3920 follow-up visits. The characteristics of the men excluded from this study were similar to those of men who were examined. The number of follow-up visits ranged from 1 to 10 (median, 2). The examination during 36.4% of the follow-up visits was performed by the same clinician who had performed the baseline examination.
At baseline, 68.9% of the men were reported as circumcised (Table 1). Circumcision was reported more commonly among men younger than 35 years (71.6%) than among men older than 35 years (59.9%). More white men were reported as circumcised (84.7%) than black (67.8%) or Hispanic (52.8%) men. Reported circumcision status was not associated with the symptoms or diagnosis of STD at the follow-up visits.
Overall, for 15.6% of the follow-up visits, the reported circumcision status differed from the status reported at the baseline visit. Disagreement was much higher among black men (18.9%) than among Hispanic (9.1%) or white (7.6%) men. Disagreement was much more likely for the men reported as uncircumcised at baseline (32.3%) than for the men reported as circumcised at baseline (8%). The amount of disagreement did not differ between scheduled and nonscheduled follow-up visits.
Men with partial circumcision may be more difficult to classify than others. One patient examined six times was reported as circumcised three times and uncircumcised three times. A follow-up examination (and photograph of unpublishable quality) showed a foreskin of intermediate length (type B or C, Figure 1). All men were analyzed with multiple follow-up examinations to determine whether clinician disagreement was more common among a small group of them. Of the 404 men with four or more examinations, 24 had at least two examinations in which circumcision status was reported differently than in the other examinations. This amount of disagreement clustering was not more than would be expected as a result of chance alone.
The race/ethnicity, gender, and academic degrees of clinicians differed by site (Table 2). Whereas four sites had several study-trained clinicians, Newark had only one, who performed 90.2% of the examinations. In the current study, 90 of the men were examined by 16 nonstudy clinicians: 8 from Newark, 6 from Denver, and 1 each from San Francisco and Baltimore.
Of 1427 follow-up examinations performed by the same clinician who performed the corresponding baseline examinations, 13.7% did not agree with their own baseline assessment. Of 2463 follow-up examinations conducted by a different clinician than at baseline, there was a 16.7% disagreement with the baseline report.
In the univariate analysis, poor reproducibility was associated with study site, circumcision status reported at baseline, the patient’s race/ethnicity, the clinician’s academic degree, the clinician’s gender, and the use of different clinicians for baseline and follow-up examinations. Reproducibility was the same for all clinicians, regardless of race/ethnicity. Poor reproducibility was most likely if the clinician was a physician’s assistant, the clinician was a woman, the patient was black, or the patient was reported as uncircumcised at baseline (Table 1). Although reproducibility was similar for registered nurses and nurse practitioners, if the clinicians at baseline and follow-up visits were both physician’s assistants, disagreement was more likely than if both were nurse practitioners (OR, 1.67; 95% CI, 1.10–2.56). Reproducibility was worse when one clinician was a woman and one was a man than when both clinicians were men (OR, 1.75; 95% CI, 1.18–2.60). Reproducibility was even lower when both clinicians were women than when both were men (OR, 2.77; 95% CI, 1.90–4.05). The associations with clinician variables remained the same after removal of Newark from the model because of its high correlation with these variables.
A significant interaction was found between a patient’s reported baseline circumcision status and the patient’s race/ethnicity: Disagreement was lowest for white men reported as circumcised at baseline (4% disagreement, 19/532 visits) and more likely for Hispanics reported as circumcised at baseline (OR, 2.34), blacks reported as circumcised at baseline (OR, 2.77), Hispanics reported as uncircumcised at baseline (OR, 3.25), whites reported as uncircumcised at baseline (OR, 12.15), and blacks reported as uncircumcised at baseline (OR, 17.08). By multivariate analysis, the clinician’s gender, the circumcision status at baseline, the patient’s race/ethnicity, and the multiplicative interaction between the baseline reported circumcision status and the patient’s race/ethnicity remained in the model. The multivariate odds ratios did not differ substantially from the univariate odds ratios (data not shown).
In this multisite study involving men of various ethnic backgrounds, a 15.6% disagreement was found between baseline and follow-up circumcision status reports. The true circumcision status could not be ascertained. Because this study did not count disagreements in which both examiners were wrong, the authors believe they underestimated the rate of actual misclassification.
Even if the observed rate is an underestimate, misclassification of this magnitude could profoundly influence the effect of association in studies that consider circumcision status a risk factor for STDs or HIV. Nondifferential misclassification of the exposure variable (in which exposure classification error is unrelated to disease status) causes the association of interest to be underestimated. 24,25 For example, if the current error rates (sensitivity of 92% because of a 8% error among the circumcised and specificity of 68% because of a 32% error among the uncircumcised) are applied to Parker et al’s 12 1983 study of circumcision and STDs, the odds ratio for gonorrhea in uncircumcised men increases from 2.1 (95% CI, 1.4–3.3) to 4.2 (95% CI, 2.5–7). 12,25 Therefore, the risk of STD or HIV infection for uncircumcised men may be higher than estimated in studies that did not consider misclassification.
In the current study, most disagreements occurred when the clinician was a woman, the patient was black, or the patient was reported initially as uncircumcised. Several hypotheses are offered to explain how clinicians’ acumen could influence the reproducibility. First, the difference between the female and male clinicians may reflect the female clinicians’ discomfort with the examination or their lack of awareness that foreskins may differ in length. The authors found no data, however, to support this hypothesis. Second, the tendency to report a man as circumcised may be influenced by the high prevalence of circumcision in the U.S. population (60–80%). 10 Therefore, a man reported as uncircumcised initially is likely to be reported as circumcised later. Third, differences in the proportion of disagreement by race/ethnicity of the patient may be related to clinicians’ perceptions of the varied prevalences of circumcision between groups of differing race/ethnicity. Fourth, some clinicians may define circumcision status differently than others on the basis of their personal formal training and experience. Finally, the nonstudy clinicians may have had a higher proportion of disagreement because, not trained specifically for this study, they may have been less careful in filling out paperwork they considered unrelated to the diagnosis of an STD.
The current study is one among a small number of studies that have examined the reproducibility of clinician-reported circumcision status by comparing multiple clinical examinations of the same patient. In a 1994 cross-sectional study of circumcision and STDs at the Seattle–King County STD clinic, the reproducibility of the data was checked by comparing clinic records for 2266 men seen by a clinician more than once between 1981 and 1989. Discrepant records for circumcision status involved only 13 (0.6%) of the men. 2 Most of the discovered errors may be explained by differences in the characteristics of the patients, clinicians, or study protocols between these two studies.
In the Seattle study, 51% of the men were white, whereas in the current study, only 17% of the men were white. White men are more likely to be circumcised than men of other races and ethnic groups in the United States. 10 In the current study, more disagreements were found in relation to black men, apparently because clinicians tended to classify them as circumcised.
There may have been differences in the gender and academic training of the clinicians who participated in the two studies, but this could not be verified. If the Seattle study used more male clinicians or physicians, this could account for a lower number of errors. Perhaps the factor accounting for most of the difference in apparent error rates is the fact that Project RESPECT clinicians were blind to the reported circumcision status at previous visits, whereas according to the Seattle researchers, that Seattle clinicians most likely were not.
The appearance of circumcised men may differ because residual foreskins differ in length. 4,6,11,14,19–21 To reduce the chances of clinicians making the wrong decision regarding a man’s circumcision status, the authors recommend that clinicians who participate in future studies of circumcision be trained to classify the lengths of foreskins rather than classify circumcision as a yes or no condition. Wynder and Licklider’s 21 four categories of male foreskins are shown in Figure 1. The clinicians in Wynder and Licklider’s 21 1960 study had particular difficulties classifying men with types B and C foreskins. They found that 13% to 23% of the men who did not claim to be circumcised had type B or C foreskins, 21 and a study of penile appearance in boys categorized 28% of the circumcised boys 3 years of age or older as having type B or C foreskins. 20
The strengths of this study include the large sample size and the multiple physical examinations of the same patients. Most studies of circumcision as a risk factor report circumcision status only once per patient. The current study was limited by the lack of variety in clinicians at each study site in terms of academic degree, race/ethnicity, and gender. The lack of ethnic and socioeconomic diversity in the study population at each site also limited this study. Another limitation was that, like most studies of circumcision status and STD or HIV infection, Project RESPECT was not designed to address the effect of circumcision status. The clinicians were not specifically trained to determine circumcision status for the purposes of this study. Instead, the study relied on each clinician’s personal medical training to determine this variable. Also, because of cultural and environmental differences in the study populations (e.g., the age at which circumcision is performed) the degree of misclassification may be different in sub-Saharan Africa, where many circumcision studies have been performed.
Disagreement was more common for female clinicians and for black patients. These associations with poor reproducibility may be clues that ultimately will help to decrease errors. Perhaps female clinicians are uncomfortable or unfamiliar with establishing circumcision status. Perhaps clinicians did not place a high priority on determining circumcision status, often guessing in ways that were influenced by the underlying prevalence of circumcision. In any case, the current analysis demonstrates the need for greater accuracy in the reporting of circumcision status in future studies.
1. Cook LS, Koutsky LA, Holmes KK. Clinical presentation of genital warts among circumcised and uncircumcised heterosexual men attending an urban STD clinic. Genitourin Med 1993; 69: 262–264.
2. Cook LS, Koutsky LA, Holmes KK. Circumcision and sexually transmitted diseases. Am J Public Health 1994; 84: 197–201.
3. Hart G. Factors associated with genital chlamydial and gonococcal infection in males. Genitourin Med 1993; 69: 393–396.
4. Moses S, Bailey RC, Ronald AR. Male circumcision: assessment of health benefits and risks. Sex Transm Infect 1998; 74: 368–373.
5. Nasio JM, Nagelkerke NJD, Mwatha A, Moses S, Ndinya-Achola JO, Plummer FA. Genital ulcer disease among STD clinic attenders in Nairobi: association with HIV-1 and circumcision status. Int J STD AIDS 1996; 7: 410–414.
6. Pepin J, Quigley M, Todd J, et al. Association between HIV-2 infection and genital ulcer diseases among male sexually transmitted disease patients in The Gambia. AIDS 1992; 6: 489–493.
7. Smith GL, Greenup R, Takafuji ET. Circumcision as a risk factor for urethritis in racial groups. Am J Public Health 1987; 77: 452–454.
8. Tyndall MW, Ronald AR, Agoki E, et al. Increased risk of infection with human immunodeficiency virus type 1 among uncircumcised men presenting with genital ulcer disease in Kenya. Clin Infect Dis 1996; 23: 449–453.
9. Weiss GN. Prophylactic neonatal surgery and infectious disease. Pediatr Infect Dis J 1997; 16: 727–734.
10. Laumann EO, Masi CM, Zuckerman EW. Circumcision in the United States: prevalence, prophylactic effects, and sexual practice. JAMA 1997; 277: 1052–1057.
11. Moses S, Plummer F, Bradley JE, Ndinya-Achola JO, Nagelkerke NJD, Ronald AR. The association between lack of male circumcision and risk for HIV infection: a review of the epidemiologic data. Sex Transm Dis 1994; 21: 201–210.
12. Parker SW, Stewart AJ, Wren MN, Gollow MM, Staton JAY. Circumcision and sexually transmissible disease. Med J Aust 1983; 2: 288–290.
13. Seed J, Allen S, Mertens T, et al. Male circumcision, sexually transmitted disease, and risk of HIV. J Acquir Immun Defic Syndr Hum Retrovirol 1995; 8: 83–90.
14. Urassa M, Todd J, Boerma JT, Hayes R, Isingo R. Male circumcision and susceptibility to HIV infection among men in Tanzania. AIDS 1997; 11: 73–80.
15. Surick I, McLaughlin M, Chiasson M, et al. HIV infection and circumcision status. New York City Department of Health, New York City, New York. In: Program and Abstracts of the International Conference on AIDS, 4–9 June 1989, Abstract T.A.P. 89.
16. Nicoll A. Routine male neonatal circumcision and risk of infection with HIV-1 and other sexually transmitted diseases. Arch Dis Child 1997; 77: 194–195.
17. De Vincenzi I, Mertens T. Male circumcision: a role in HIV prevention? [editorial review]. AIDS 1994; 8: 153–160.
18. Van Howe RS. Circumcision and infectious diseases revisited. Pediatr Infect Dis J 1998; 17: 1–6.
19. Lilienfeld AM, Graham S. Validity of determining circumcision status by questionnaire as related to epidemiological studies of cancer of the cervix. J Natl Cancer Inst 1958; 21: 713–720.
20. Van Howe RS. Variability in penile appearance and penile findings: a prospective study. Br J Urol 1997; 80: 776–782.
21. Wynder EL, Licklider SD. The question of circumcision. Cancer 1960; 13: 442–445.
22. Schlossberger NM, Turner RA, Irwin CE. Early adolescent knowledge and attitudes about circumcision: methods and implications for research. J Adolesc Health 1991; 12: 293–297.
23. Kamb ML, Fishbein M, Douglas JM Jr, et al. Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases: a randomized controlled trial. JAMA 1998; 280: 1161–1167.
24. Mertens TE. Estimating the effects of misclassification. Lancet 1993; 342: 418–421.
25. Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic Research: Principles and Quantitative Methods. New York: Van Nostrand Reinhold, 1982.