Broad, Jennifer M. MPH*; Manhart, Lisa E. PhD*; Kerani, Roxanne P. PhD†; Scholes, Delia PhD‡; Hughes, James P. PhD*; Golden, Matthew R. MD, MPH*†
Screening of young women is the centerpiece of the US chlamydia control program. The value of screening is supported by randomized trials demonstrating that routine testing of young women can reduce the occurrence of pelvic inflammatory disease and by mathematical models suggesting that the intervention can decrease chlamydial prevalence.1–6 However, little empiric evidence exists to demonstrate that chlamydial screening affects chlamydial incidence or prevalence, and mathematical models suggest that the effect of screening is highly dependent on the level of coverage achieved.7,8 At present, health departments have little ability to monitor this outcome. The absence of such data limits public health efforts to improve levels of screening or even recognize whether screening efforts are stable, improving, or deteriorating.
Most US chlamydial screening coverage estimates have relied on a measure developed through the Healthcare Effectiveness Data and Information System (HEDIS).9–13 The HEDIS measure is widely used by managed care organizations to assess quality of care and compare care across organizations. In the absence of alternatives, HEDIS estimates have sometimes been used as measures of population-level screening coverage.14,15 However, these estimates are not population based, do not capture out-of-plan care and, in at least some instances, suffer from inaccuracies in the identification of testing events and the definition of the sexually active population.13,16 To address some of these limitations, Centers for Disease Control and Prevention (CDC) investigators developed an indirect method to estimate screening coverage that defines the number of tests performed in an area by dividing the number of reported cases in the jurisdiction by the estimated positivity in tested women.17,18 Although this approach is simple, it is sensitive to variations in the estimate of chlamydial positivity among tested women, which is not precisely known, and results in an estimate of number of tests performed, not the number of women tested.
We compared 9 variations of HEDIS and indirect estimates of chlamydial screening coverage and explored how estimates vary with different methods and assumptions affecting the size of the eligible population and the number of tests performed. Our goals were to understand how much estimates might vary depending on the methods and data used and to define a standard method that health departments might use to monitor chlamydial screening.
We used 3 methods to estimate chlamydial screening coverage among sexually active women aged 15 to 25 years in Washington State in 200919,20: (1) screening coverage within a sentinel population (i.e., Group Health Cooperative) using several variations of the HEDIS measure; (2) indirect estimation, which was based on the number of cases of chlamydial infection among women reported to the Washington State Department of Health and estimates of test positivity derived from laboratories in the state; and (3) direct-indirect estimation, which combined a direct measure of the number of tests performed to identify cases reported from laboratories providing testing volume and positivity data with an indirect estimate of the number of tests performed to find the remaining reported cases. Table 1 summarizes how we defined the sexually active population (the denominator) and the number of women tested for chlamydial infection (the numerator) for each of our 9 estimates.
Our sentinel estimation method sought to define chlamydial screening coverage among women enrolled and receiving care in Group Health, a mixed model–managed care system in Washington State. Group Health has well-developed computerized data systems to monitor laboratory testing, hospitalizations, ambulatory care, and pharmacy prescription fills, and data from these systems have been used previously to monitor chlamydial screening and the occurrence of pelvic inflammatory disease.21 We calculated 4 estimates for this population. All 4 defined the number of women tested for Chlamydia trachomatis (the numerator) based on the presence of any laboratory claim for a chlamydial test in 2009, the standard HEDIS numerator. The 4 estimates reflected 2 sources of variation in the definition of the population eligible for screening (the denominator): (1) inclusion only of users of Group Health care (the current method) versus inclusion of all Group Health enrollees and (2) estimates of the size of the sexually active population based on current HEDIS methods versus National Survey of Family Growth (NSFG) data. We elected to estimate screening coverage among all enrollees and not just users because we assumed that at least some sexually active women enrolled in Group Health aged 15 to 25 years did not use care in 2009. Although some of these women may have been tested for C. trachomatis outside Group Health, focusing exclusively on users likely inflates screening coverage estimates; our estimates among all enrollees were designed to assess how much the focus on users might affect the HEDIS measure.
In defining the Group Health population eligible for chlamydial screening, we first defined the number of women aged 15 to 25 years enrolled in Group Health (i.e., enrollees). This included women with continuous Group Health coverage in 2009 (i.e. no lapse in coverage >1 month), including persons who did not access services. We then limited this enrollee population to women who received medical care or who had a claim submitted to Group Health (i.e., users).
We used 2 approaches to define women who were sexually active. First, we used HEDIS denominator criteria.22 These criteria draw on more 600 International Classification of Diseases, Ninth Revision (ICD-9) laboratory, procedure, diagnosis, treatment, and prescription codes from submitted claims to identify services related to pregnancy, contraception, and/or sexually transmitted diseases (STDs). Women who receive such services in a 12-month period are defined as sexually active. Second, we estimated the proportion of women aged 15 to 25 years who were sexually active using national data from the 2006 to 2008 NSFG. National Survey of Family Growth is an interview-based survey that collects data from a probability sample of US men and women aged 15 to 44 years.23 We defined the proportion of the female population that was sexually active as the proportion of all female NSFG participants aged 15 to 25 years reporting at least 1 sex partner in the past year.17,24
Thus, we defined a total of 4 denominators: (1) user HEDIS—the standard HEDIS population including only users with sexual activity defined using ICD-9 codes; (2) enrollee HEDIS—Group Health enrollees including nonusers, with the number of sexually active enrollees defined using HEDIS criteria among users and NSFG data among nonusers; (3) user NSFG-HEDIS—users with the sexually active population defined using NSFG sexual activity estimates; and (4) enrollee NSFG-HEDIS—all enrollees including nonusers, with the size of the sexually active population defined using NSFG.
Indirect and Direct-Indirect Estimation
We derived indirect estimates of screening coverage using a modification of the approach developed by Levine et al.17 (Table 2, basic equation-EQ1). All indirect estimates defined the size of the population eligible for screening (the denominator) by multiplying the national NSFG estimate of the proportion of 15- to 25-year-old women who are sexually active by the number of women aged 15 to 25 years in Washington, based on the 2010 US census. We calculated 3 indirect screening coverage estimates. All 3 used surveillance data to define the number of diagnosed cases of chlamydial infection in women aged 15 to 25 years in Washington State. That number was then divided by 3 different estimates of test positivity (i.e., the proportion of tests performed in women that were positive) to yield estimates of number of tests performed. Those estimates were then divided by an estimate of the number of tests performed per tested woman to define estimates of the number of women tested (the numerator). Table 3 presents test positivity data from our 3 sources of laboratory data: Group Health, the Washington Infertility Prevention Project (IPP), and aggregate data from 4 commercial laboratories. Infertility Prevention Project was a CDC-funded program that supported chlamydial testing in settings such as primary care, family planning, STD, and school-based clinics. In some clinics, only a subset of specimens collected for chlamydial testing—typically specimens obtained from uninsured patients—were sent to IPP laboratories. Data from commercial laboratories were obtained as part of a public health effort to better define chlamydial screening in the state; Public Health–Seattle and King County contacted laboratory directors of the 5 largest commercial laboratories in Washington (excluding Group Health, which provided data outside this process) and asked them for chlamydia positivity data. Four laboratories provided either aggregate data on the total number of tests performed in women aged 15 to 25 years in 2009 and test positivity or de-identified line-listed data on all tests for C. trachomatis performed in their laboratory in that year (Table 3). All data were limited to tests ordered by providers in medical facilities with Washington State zip codes. Three of the reporting laboratories also provided data on the number of tests performed per woman tested in 2009, which we averaged to obtain the mean number of tests per tested woman in a year (NumTest in the equation).
A fourth indirect estimate adjusted for reason for chlamydial testing (Table 2, EQ2). We developed this adjusted estimate because 2009 Washington IPP data indicated that test positivity varied based on reason for testing: 8% among symptomatic women, 30% among women tested as contacts to a partner with chlamydial infection, and 6% among screened women. All case reports in Washington include fields in which medical providers report the reason for STD testing using the above categories. Because reason for testing among women testing negative was only available for tests done through IPP, we calculated this adjusted estimate using only IPP data.
Finally, to use all available data and minimize the influence of our test positivity estimate on our screening coverage, we developed a combined direct-indirect estimate of screening coverage (Table 2, EQ3). This measure directly defined the number of tests performed to identify cases diagnosed through reporting laboratories and estimated the number of tests performed in nonreporting laboratories based on a weighted average of test positivity in laboratories that provided data for the analysis.
A total of 48,054 women aged 15 to 25 years were enrolled in Group Health in 2009. Of these women, 37,595 (78%) used services. Among users, 22,679 (60.3%; 95% confidence interval [CI], 59.8%–60.8%) had utilization or claims data that met HEDIS criteria defining them as sexually active. This estimate was almost identical to the estimate of 61.1% (95% CI, 56.5%–65.5%) derived using NSFG data.
Sentinel population screening coverage estimates among Group Health users and enrollees are presented in Table 4. Estimates varied from 34.2% to 44.5%. Screening coverage estimates for all enrollees were approximately 10 percentage points lower than estimates restricted to users (34.2%–34.8% vs.44.6%–44.5%). These estimates did not substantially vary based on whether NSFG data, HEDIS criteria, or a combination of both was used to define the size of the sexually active population.
Indirect estimates of chlamydial screening coverage varied from 46.4% to 68.7%, depending on the source of positivity data used to estimate the number of women tested. Among laboratories providing data, the highest prevalence of infection was observed in the population tested through IPP, and the screening coverage estimate based on IPP data (49.1%) was consequently lower than those based on other data sources. At the other extreme, the prevalence of infection among women tested through Group Health was only 4.5%, and the population screening coverage estimate derived using those data was the highest ofthe estimates we defined (68.7%). Adjusting the IPP estimate for reason for testing had little effect on our measure of screening coverage, decreasing the estimate based on the IPP positivity from 49.1% to 46.4%.
The direct-indirect estimate, which added a direct measure of the number of tests performed to identify 5812 (52%) of the 11,218 total cases of chlamydial infection reported in young women in Washington in 2009 (103,752 tests) to an indirect estimate of the number of tests performed to identify the remaining 48% of reported cases from unknown laboratories (105,113 tests), yielded a screening coverage estimate of 57.6%.
We found that HEDIS estimates of chlamydial screening coverage were substantially lower than indirect estimates calculated using data on the number of infections reported in Washington State and laboratory data on test positivity. Indirect estimates of screening coverage varied widely based on the source of data used to define test positivity. However, because we were able to obtain data from laboratories that diagnosed more than half of all chlamydial infections in young Washington State women, we believe that our direct-indirect estimate is the best measure of population-level screening coverage. The direct-indirect approach minimizes potential error by directly measuring the number of tests performed to identify a large proportion of all cases and derives positivity data from diverse sources, diminishing error resulting from the use of data gathered from a potentially nonrepresentative population served by a single laboratory. Based on this approach, we estimate that almost 58% of all sexually active women aged 15 to 25 years in Washington were tested for C. trachomatis in 2009, a number that exceeds most prior screening coverage estimates.10,17
That our indirect estimates of screening coverage were higher than most10 but not all14 past HEDIS estimates was surprising and encouraging. We had initially hypothesized that screening coverage in a well-organized health maintenance organization would be higher than that observed in the general population. Several factors may explain the disparity we found between our HEDIS and indirect estimates. First, screening may, in fact, be lower in Group Health than in the general population of Washington young women. Alternatively, HEDIS may underestimate screening because of its failure to include tests performed outside the health care plan for which no claim is submitted (i.e., if a woman is tested outside Group Health and no claim is submitted to that organization, the tested woman would be misclassified as untested). This issue merits further study. Healthcare Effectiveness Data and Information System may also underestimate screening coverage by overestimating the proportion of women who are sexually active, although the close agreement we observed between HEDIS and NSFG-based estimates of sexual activity suggests that this is not a large source of error. Finally, our indirect estimates of screening coverage may be too high. However, we believe that this final possibility is unlikely. Although surveillance data may underestimate the number of cases of chlamydial infection in the state, thereby leading to an underestimate of screening coverage, it seems unlikely that surveillance overestimates the number of chlamydial diagnoses. If chlamydia positivity in laboratories for which we had no data were considerably higher than in the laboratories included in our analysis, our estimates of screening coverage would be too high. However, even if one assumes that nonreporting laboratories had a positivity of 6.3%, the highest positivity we observed among reporting laboratories, our overall direct-indirect estimate of screening coverage would only decline from 58% to 52%.
Although data assessing HEDIS in other areas would be useful, our findings suggest that the HEDIS measure may not provide an accurate absolute measure of screening coverage, either for all of a plan’s enrollees or for the wider population. However, the problem the HEDIS measure was originally designed to address—the need for a common metric for assessing quality of care within and between managed care organizations—persists. The challenge is how to make the measure more accurate and how to balance sources of error that inflate screening coverage (e.g., the focus on users rather than all enrollees) with those that may diminish coverage estimates (e.g., failure to capture out-of-plan care). One approach to improving the accuracy of HEDIS might be to routinely calculate screening coverage estimates for both users and all enrollees. These 2 metrics would give a fuller picture of quality of care and would identify the extent to which high screening coverage in some instances may result from plans providing women no medical services at all.
Our results suggest that chlamydial screening coverage is likely higher than previously estimated and that the existing program has enjoyed substantial success, at least as measured by its reach into the population compared with other recommended screening procedures. Although published estimates have used varying age ranges, limiting their direct comparability to our findings, using the HEDIS method, CDC investigators estimated that 41.7% of sexually active US women aged 16 to 24 years were tested for C. trachomatis in 2007.10 More recent HEDIS data suggest that coverage is close to 50% among Medicaid enrollees, but only 43% among persons enrolled in commercial health plans.14 Of note, screening coverage achieved through what is sometimes termed “opportunistic screening”25 in Washington State seems to be substantially higher than that achieved in European nations with national chlamydial screening programs. Recent estimates suggest that 13% of young men and women in Sweden, 14.6% to 25.2% of young women in England, and 25% of sexually active young women in the Netherlands are tested for C. trachomatis annually.26–28 Chlamydial screening coverage among young women in Washington also seems to be comparable with screening coverage for several other commonly recommended preventive medical services, such as colonoscopy (50.2%), mammography (53.0%), and prostate exams (44.1%), although the percentage of women who receive Papanicolaou tests (78.3%) is higher than the percentage tested for chlamydial infection.29
Strengths of our study include our use of data from a large health maintenance organization and from several laboratories that, in aggregate, identified 52% of all cases of C. trachomatis in Washington during the study period; our ability to explore how different approaches to defining the sexually active population and estimating test positivity affect screening coverage estimates; and the use of data on the number of tests performed per woman to adjust for the fact that some women test for chlamydial infection more than once annually.
Our study also had several limitations. First, laboratory test positivity data used in the study may not have been representative of all laboratories in Washington State, and our estimates of screening coverage varied substantially based on the source of laboratory positivity data used. This finding highlights the importance of obtaining laboratory data from multiple sources when using indirect estimation. Second, we only calculated a HEDIS measure for Group Health, which may be not be representative of other large health care providers. Third, several of our estimates used national NSFG data to define the proportion of women aged 15 to 25 years who were sexually active. It is possible that young women in Washington State are more or less sexually active than women nationally. National Survey of Family Growth does not enroll a large-enough sample to create a precise state-specific estimate of sexual activity. However, within NSFG, there are no significant differences in levels of sexual activity among young women by region (G. Tao, personal communication, CDC). Finally, our findings may not be generalizable to other areas of the United States.
In summary, we found that the HEDIS measure of chlamydial screening may underestimate true population-level screening coverage and that indirect estimation using data collated from large laboratories is feasible and is probably the preferred method for estimating population-level screening coverage. Based on these findings, we believe that state and local health departments should work with large laboratories to obtain de-identified data to follow trends in chlamydial screening. Because laboratory data do not typically include demographic information such as race and ethnicity, additional efforts may be requiredto identify sectors of the population with low screening coverage.30 On the other hand, insofar as collection of laboratory data on chlamydial testing can be linked to acquisition of information on other tests (e.g., HIV and hepatitis C testing), the approach we propose has the further advantage of allowing health departments to monitor a variety of communicable diseases of public health significance.
1. Scholes D, Stergachis A, Heidrich FE, et al.. Prevention of pelvic inflammatory disease by screening for cervical chlamydial infection. N Engl J Med 1996; 334: 1362–1366.
2. Ostergaard L, Andersen B, Moller JK, et al.. Home sampling versus conventional swab sampling for screening of Chlamydia trachomatis in women: A cluster-randomized 1-year follow-up study. Clin Infect Dis 2000; 31: 951–957.
3. Oakeshott P, Kerry S, Aghaizu A, et al.. Randomised controlled trial of screening for Chlamydia trachomatis to prevent pelvic inflammatory disease: The POPI (Prevention of Pelvic Infection) trial. BMJ 2010; 340: c1642.
4. Kretzschmar M, Welte R, van den Hoek A, et al.. Comparative model-based analysis of screening programs for Chlamydia trachomatis infections. Am J Epidemiol 2001; 153: 90–101.
5. Turner KM, Adams EJ, Lamontagne DS, et al.. Modelling the effectiveness of chlamydia screening in England. Sex Transm Infect 2006; 82: 496–502.
6. Low N, McCarthy A, Macleod J, et al.. Epidemiological, social, diagnostic and economic evaluation of population screening for genital chlamydial infection. Health Technol Assess 2007; 11:iii–iv, ix–xii, 1–165.
7. Althaus CL, Heijne JC, Roellin A, et al.. Transmission dynamics of Chlamydia trachomatis affect the impact of screening programmes. Epidemics 2010; 2: 123–131.
8. Regan DG, Wilson DP, Hocking JS. Coverage is the key for effective screening of Chlamydia trachomatis in Australia. J Infect Dis 2008; 198: 349–358.
9. Ray MN, Wall T, Casebeer L, et al.. Chlamydia screening of at-risk young women in managed health care: Characteristics of top-performing primary care offices. Sex Transm Dis 2005; 32: 382–386.
10. Centers for Disease Control and Prevention (CDC). Chlamydia screening among sexually active young female enrollees of health plans—United States, 2000–2007. MMWR Morb Mortal Wkly Rep 2009; 58: 362–365.
11. Shih SC, Bost JE, Pawlson LG. Standardized health plan reporting in four areas of preventive health care. Am J Prev Med 2003; 24: 293–300.
12. Mangione-Smith R, McGlynn EA, Hiatt L. Screening for chlamydia in adolescents and young women. Arch Pediatr Adolesc Med 2000; 154: 1108–1113.
13. Wei F, Walsh CM. Validation of data collection for the HEDIS performance measure on chlamydia screening in an MCO. Am J Manag Care 2003; 9: 585–593.
14. Centers for Disease Control and Prevention (CDC). CDC Grand Rounds: Chlamydia prevention: Challenges and strategies for reducing disease burden and sequelae. MMWR Morb Mortal Wkly Rep 2011; 60: 370–373.
15. National Committee for Quality Assurance. The State of Health Care Quality. Washington, DC: National Committee for Quality Assurance, 2010.
16. Tao G, Walsh CM, Anderson LA, et al.. Understanding sexual activity defined in the HEDIS measure of screening young women for Chlamydia trachomatis. Jt Comm J Qual Improv 2002; 28: 435–440.
17. Levine WC, Dicker LW, Devine O, et al.. Indirect estimation of Chlamydia screening coverage using public health surveillance data. Am J Epidemiol 2004; 160: 91–96.
18. Tao G, Tian LH, Peterman TA. Estimating chlamydia screening rates by using reported sexually transmitted disease tests for sexually active women aged 16 to 25 years in the United States. Sex Transm Dis 2007; 34: 180–182.
19. Meyers DS, Halvorson H, Luckhaupt S. Screening for chlamydial infection: An evidence update for the U.S. Preventive Services Task Force. Ann Intern Med 2007; 147: 135–142.
20. Workowski KA, Berman SM. Sexually transmitted diseases treatment guidelines, 2006. MMWR Recomm Rep 2006; 55: 1–94.
21. Scholes D, Satterwhite CL, Yu O, et al.. Long-term trends in Chlamydia trachomatis infections and related outcomes in a U.S. managed care population. Sex Transm Dis 2012; 39: 81–88.
22. National Committee for Quality Assurance. HEDIS Effectiveness of Care 2010, Prevention and Screening. Washington, DC: National Committee for Quality Assurance, 2009: 31–34.
23. Lepkowski J, Mosher W, Davis K, et al.. The 2006–2010 National Survey of Family Growth: Sample Design and Analysis of a Continuous Survey. Vital Health Stat. Vol. 2. Hyattsville, MD: National Center for Health Statistics, 2010.
24. Abma J, Martinez G, Copen C. Teenagers in the United States: Sexual Activity, Contraceptive Use, and Childbearing, National Survey of Family Growth 2006–2008. Vital Health Stat. Vol. 23. Hyattsville, MD: National Center for Health Statistics, 2010.
25. Low N. Screening programmes for chlamydial infection: when will we ever learn? BMJ 2007; 334: 725–728.
26. Low N. Current status of chlamydia screening in Europe. Euro Surveill 2004; 8: 5.
27. Op de Coul EL, Gotz HM, van Bergen JE, et al.. Who participates in the Dutch chlamydia screening? A study on demographic and behavioral correlates of participation and positivity. Sex Transm Dis 2012; 39: 97–103.
28. Sheringham J, Simms I, Riha J, et al.. Will chlamydia screening reach young people in deprived areas in England? Baseline analysis of the English National Chlamydia Screening Programme delivery in 2008. Sex Transm Dis 2011; 38: 677–684.
29. Smith RA, Cokkinides V, Brooks D, et al.. Cancer screening in the United States, 2011: A review of current American Cancer Society guidelines and issues in cancer screening. CA Cancer J Clin 2011; 61: 8–30.
30. Datta SD, Torrone E, Kruszon-Moran D, et al.. Chlamydia trachomatis trends in the United States among persons 14 to 39 years of age, 1999–2008. Sex Transm Dis 2012; 39: 92–96.