Skip Navigation LinksHome > November 2008 - Volume 14 - Issue 6 > Availability of Data to Measure Disparities in Leading Healt...
Journal of Public Health Management & Practice:
doi: 10.1097/01.PHH.0000338385.82436.44
Original Article

Availability of Data to Measure Disparities in Leading Health Indicators at the State and Local Levels

Gold, Marsha ScD; Dodd, Allison Hedley PhD; Neuman, Melissa MS

Free Access
Article Outline
Collapse Box

Author Information

Marsha Gold, ScD, is Senior Fellow, Mathematica Policy Research, Inc., Washington, District of Columbia.

Allison Hedley Dodd, PhD, is Health Researcher, Mathematica Policy Research, Inc., Washington, District of Columbia.

Melissa Neuman, MS, is Health Research Analyst, Mathematica Policy Research, Inc., Washington, District of Columbia.

Corresponding Author: Marsha Gold, ScD, Mathematica Policy Research, Inc., 600 Maryland Ave SW, Suite 550, Washington, DC 20024 (Mgold@Mathematica-MPR.com).

This study was supported under a grant from a special solicitation for Public Health Systems Research through the Robert Wood Johnson Foundation's Healthcare Financing and Organization Initiative, which is administered by AcademyHealth. Sharon Arnold, Kate Papa, and others at AcademyHealth provided valuable advice in pursuing this project and helped obtain feedback from others likely to use this work. Peggy Honore of the Office of the Assistant Secretary for Health and Office of Minority Health at the US Department of Health and Human Services encouraged us to write this article, and particularly to pursue the question of available substate data. At Mathematica, Leslie Conwell, a member of the broader project team, provided insight from the field on gaps in data on health disparities, as viewed by users, and Melanie Besculides provided additional feedback. Felita Buckner provided secretarial support. Jim and Molly Cameron edited the document.

Collapse Box

Abstract

Objectives: Healthy People 2010 identifies the elimination of health disparities as a critical national goal. The article analyzes the availability of state and local data to support this work.

Methods: We assessed data availability for the 10 leading health indicators (LHIs), comprising a set of 26 measures. Our analysis is based on a mid-2007 review of federal and state Web sites.

Findings: Federal data sources allow aggregate state estimates for 24 LHI measures, although some either are not available for all states or vary from the federal definition. National sources capture some but not all of the subgroup characteristics, defined as national disparities priorities. Limited sample size is a barrier to generating state estimates for specific subgroups, and data by geographic subdivision within a state are often lacking. States also vary in how aggressively they use disparities data or make them available externally.

Conclusions: Federal leadership has been critical to state capacity to assess LHI disparities. Although some relevant state-level disparities data exist, major gaps remain, local estimates are limited, and some states make better use of the data than others. Continued federal leadership and support is critical to states' abilities to address Healthy People 2010's disparities goal.

Healthy People 2010 (HP 2010) identifies the elimination of health disparities as one of two overall national goals.1 HP 2010 is concerned with disparities associated with a broad range of characteristics—sex, race, ethnicity, income, education, disability, and sexual orientation.

State and local data on health disparities are vital to support both the assessment and the planning and development of effective interventions to improve health and eliminate such disparities. The types of state and local analyses necessary to identify and, even more critically, intervene to eliminate disparities put additional demands on public health data and national infrastructure. Through the National Committee on Vital and Health Statistics, the Centers for Disease Control and Prevention (CDC) Assessment Initiative, and other entities, the federal government has worked with states for many years to increase and strengthen the capacity of public health data systems to collect, report, analyze, and disseminate data on disparities at the national, state, and local levels, particularly for various ethnic, racial, and linguistic populations.2–4 This article complements recent analysis of national data with an analysis of the availability of these national and state data to those seeking to address disparities at the state level and in smaller geographic areas within states.5

Back to Top | Article Outline

Study Focus and Methods

Our assessment examines the availability of disparities data at the state and local levels for the 10 leading health indicators (LHIs) identified in HP 2010. Although state and local health priorities may differ from national priorities, we chose the LHIs (and the 26 separate measures they encompass) as an evaluation framework because they cover a range of broad public health issues identified by experts as reflective of the health concerns of the United States (Table 1).1 Because the experts considered data availability among other factors in selecting the LHIs, the findings may overestimate the overall availability of data on disparities at the state and local levels. We defined each of the LHIs using the HP 2010 documentation and accessed a combination of federal and state Web sites for our availability assessment.* For the sake of efficiency, we began with federal sites because LHIs typically can be constructed from federal sources. We reviewed the CDC Wonder database to create a preliminary summary of indicators that were available for states, either in their national form or via alternative sources or specifications suggested by the CDC.6 We then updated the analysis using the most recent state-level data available on federal Health and Human Services (HHS) Web sites for relevant sources, including the Behavioral Risk Factor Surveillance System (BRFSS), the Healthcare Cost and Utilization Project (HCUP), the National Immunization Survey (NIS), the National Survey on Drug Use and Health (NSDUH), the National Vital Statistics System (NVSS), and the Youth Risk Behavior Surveillance System (YRBSS). To identify other data sources and update federal data, we compared federal information with information available on state Web sites. The state review included the states' departments of health, public health, and/or substance abuse, as well as mental health Web sites. We also conducted more general Internet searches using common terms such as “health data,” “health survey,” “immunization registry,” and “Healthy 2010.” The analysis is current as of summer 2007. Although our methods do not provide a complete accounting for each state, the findings are the most comprehensive and consistent examination of this kind of data availability of which we are aware. (Readers seeking additional detail on methods and findings can review Dodd et al.7) Finally, to learn more about state activities related to health disparities, we reviewed salient documents and contacted several individuals knowledgeable in this area to obtain their feedback on initiatives underway in states.

Table 1
Table 1
Image Tools
Back to Top | Article Outline

Findings

Historical efforts have enhanced LHI measures for states

State estimates for LHIs are available for 24 of the 26 measures associated with the 10 LHIs. The two exceptions are treatment of recognized depression and a biomarker for exposure of nonsmokers to environmental tobacco smoke. However, a number of gaps exist in many state-level LHI estimates. For example, seven other measures can be constructed at the state level, but with some limitations. Four measures specified nationally are available only for a subset of the state population. State-level access to care measures (coverage and source of care) are available only for adults, not the population*, and state substance abuse measures for teens cover grades 9 to 12, a smaller age range than the national measures (12–17 years). The LHI for illicit drug use can be supported at the state level only by pooling 2 years of data. In 2005, the BRFSS questions about condom use (relevant to two measures) were moved from the core questionnaire to the family planning module. Between 2004 and 2006, the number of states with data from the BRFSS for these measures declined from 51 (all 50 and the District of Columbia) to 7.

The data for the LHI measures come from seven different national sources; all but one are maintained by the HHS, and most involve extensive state participation and collaboration (Table 2). Two sources (the HCUP and the YRBSS) do not include all states because they depend on voluntary participation; furthermore, not all HCUP participants have adequate data to examine disparities. The BRFSS, the YRBSS, and the NVSS are particularly important for state estimates of LHIs because they account for all but four of the available measures. The BRFSS and the NVSS provide annual data, but the YRBSS is fielded only every other year. Only 43 states participate in the YRBSS; the survey also covers only students in grades 9 to 12 (as opposed to all adolescents, as with the national indicators).

Table 2
Table 2
Image Tools

Some states have elected to supplement federal data collection efforts. Forty-three states operate immunization registries. Among state surveys, the most common are tobacco surveys covering adults (n = 24) or youth (n = 25). States may also field their own health interview (n = 7), health insurance (n = 7), and state-sponsored youth surveys (n = 13), some of which substitute for the YRBSS. As the BRFSS designers probably intended, states seem more likely to expand their participation in the BRFSS than to mount their own independent surveys, with the associated investments; tobacco surveys are an exception.

State-sponsored surveys do not provide the national uniformity of federal surveys and place greater analytical demands on states, but states that sponsor them may do so to provide them with the ability to capture additional information and gain flexibility and analytical depth. Such flexibility could involve larger sample sizes or more intricate sample designs that better reflect a state's geography or population, as well as questions that may not be included on federal surveys (eg, sexual orientation). Two state surveys measure LHIs that cannot be gleaned from federal sources (LHI 18-9b, treatment for recognized depression). The ambitious and large California Health Interview Survey, modeled on the National Health Interview Survey, seeks to support estimates for a large number of specific ethnic groups that comprise California's multicultural population, as well as provide county-level estimates for all or most counties.9

Back to Top | Article Outline
Key sources capture some but not all data needed to measure disparities

To analyze disparities in LHIs at any geographic level, data systems must include information on relevant subgroup characteristics. National sources relevant to LHIs capture data on some but not all subgroup characteristics identified for attention in HP 2010 (see Table 2). Each relevant federal source captures age, sex, and race/ethnic subgroup, none captures sexual orientation in a standard question, and only the BRFSS captures disability; however, the questions are limited.* Income and education are not captured by the HCUP, the NSDUH, or the YRBSS. The NVSS gathers information on education but not income. The BRFSS, the source for 11 measures of LHIs, captures each subgroup characteristic except sexual orientation. (Two states—Massachusetts and Washington—added this to their surveys in 2006.) Beyond these omissions, there are quality issues with some of the subgroup data. For example, the NVSS must rely on informants that may not always be very knowledgeable to report on the race/ethnicity and other characteristics of the deceased.10

Back to Top | Article Outline
Other barriers limit states' ability to construct and use LHIs to address disparities

Even when indicators are measured and subgroup identifiers exist, there are barriers that limit the availability of LHI measures to address disparities. The barriers discussed here involve (1) limited power, (2) relevance, and (3) ability to support estimates for localities.

Back to Top | Article Outline
Limited power

Because many of the major sources used by states to develop LHIs rely on sample surveys, sample size and the population size of various subgroups within states have a major effect on the estimates that can be created, as well as on their precision.

To gain insight on power as a constraint, we examined the BRFSS and selected other federal sources for data on race/ethnic subgroups within each state against the CDC guidelines for data suppression.6 The BRFSS seeks to interview a minimum of 4000 adults in each state, but actual samples vary with federal and state funding; 2005 sample sizes ranged from 2793 to 23256, taking into account both federal and state support. The CDC guideline is that estimates should not be calculated with a cell size of fewer than 50 respondents.11 They use a relatively broad four-category subgroup set (non-Hispanic Whites, non-Hispanic Blacks, Hispanics, and non-Hispanic “other” races). Of the 50 states and the District of Columbia, 1 could not support estimates for Hispanics (Maine) and 10 did not have sample sizes large enough to support estimates of non-Hispanic Blacks (Alaska, Idaho, Maine, Montana, New Hampshire, North Dakota, South Dakota, Utah, Vermont, and Wyoming). Nine of this subgroup had very low prevalence of non-Hispanic Blacks in the state population. Two of these states compensated by pooling multiple years' data, four noted the omission of the subgroup, and four did not present any race/ethnic data on their Web sites. Three states could not report for the category “non-Hispanic other” (Alabama, Iowa, and West Virginia). Similar analysis of YRBSS data for 2003/2005 shows that for the 43 relevant states, 28 were able to report survey estimates for at least one non-White racial and ethnic subgroup on the basis of the YRBSS standard (≥100).12 On the NIS, only 17 states were able to report for non-Hispanic White or any other subgroup; only Tennessee could report for more than one additional group.

Back to Top | Article Outline
Relevance

Although LHI estimates by subgroup may exist, they may not be the estimates users want or need. In interviews with staff working on disparities within the HHS and in several leading states, we heard reports of perceived gaps between the available data and the information sought to address disparities. Interviewees said, for example, that states tend to report aggregated data (eg, non-White versus White), which does not help in the assessment of disparities for individual subgroups of interest, such as specific subgroups by national origin. Because many public health interventions tend to be conducted locally, statewide estimates could lack the precision to identify areas within the state where disparities are most problematic, which programs are needed to address particular subgroups (eg, Massachusetts residents from the Dominican Republic vs Puerto Rico; the two groups have different risk profiles), or whether the intervention worked. When initiatives are spurred by interest in a particular disease, sample size may be inadequate to support studies of disparities within a subgroup. Although the CDC has made available estimates of seven LHIs for 153 metropolitan/micropolitan statistical areas in the BRFSS SMART (Selected Metropolitan/Micropolitan Area Risk Trends) program, these data do not include race/ethnicity or other subgroup detail. When we looked at state Web sites, only 19 reported county- or region-level BRFSS estimates, many of them composites of multiple years. (We did not examine which of these included separate estimates by subgroup.)

Back to Top | Article Outline
Ability to support estimates for geographic subdivisions

Data sources such as the NVSS and the HCUP are census-type datasets. Unless privacy protections are triggered because of small numbers, such sources are able to support estimates for individual subdivisions because they include all relevant events.* Data sources that rely on surveys are more constrained in the ability to provide such detailed geographic estimates because of survey design. The BRFSS estimates depend on state sample decisions and, at best, appear to support substate estimates for select metropolitan areas and counties. Substate data through the YRBSS were available only for 21 school districts in 2005. The NIS provides estimates for 28 select urban areas, with 19 states giving some substate breakdown on their Web sites. The NSDUH produces estimates for substate treatment planning areas, which could be relevant to disparities interventions, but multiple years of data must be pooled to support such estimates, so it is difficult to track change.

Back to Top | Article Outline
States vary in their ability to overcome these barriers

The availability of data to support state and local initiatives depends not only on what is collected or reported nationally but also on the ways in which states make use of their existing data. Targeted data supplementation, creative analysis, and effective presentations allow more comprehensive reporting on health issues of interest and effective communication. In our analysis of the activity reflected in statewide sites, we found considerable variation in the supplemental state-specific data collected, how they are reported, and what techniques are used to make such data useable to a broad cross-section of stakeholders (Table 3).

Table 3
Table 3
Image Tools
Table 3
Table 3
Image Tools

Our analysis showed wide variation in practices across states. For example, states differed in the timeliness with which data were reported. Although all states publish vital statistics online, and often more currently than are reported nationally, only 20 had posted data for 2005 or later at the time of our mid-2007 review. While two states had 2006 data, two others had nothing more recent than 2000. Forty-three states had some county-level vital statistics online, although a few used only pooled data. To complement what the HHS publishes, 46 of the 50 states and the District of Columbia publish BRFSS data on their own Web sites; in 2007, 14 had published the most recent survey (2006). If county data on the BRFSS were reported, it typically was older and based on pooled analysis. The YRBSS is made available online in 38 of the 43 states, but these data are sometimes posted on education instead of public health Web sites. The lack of posting does not necessarily mean that data are not available and used, but it does mean that they probably are not available to a broad spectrum of users in that state.

States also differed in the way data were made accessible to users. Twenty-eight states made some public health data available in an interactive database, six of these with an interactive mapping or geographic information system (GIS) component. Although vital statistics typically were included, many also included BRFSS or YRBSS data or the results of state-specific surveys. California, Maine, and Washington included their own state-sponsored surveys in interactive databases. Interactive portals are useful because they give users the opportunity to create their own tabulations of data from the subgroup information available on the Web site. These sites also typically provide a single online point of access to different public health datasets, making these data more convenient for users to find and analyze.

Many states also parallel federal processes and create their own State Healthy People reports, providing benchmarks and goals for a variety of health indicators. About half of the states (n = 26) had published such a report within the 5 years prior to our review (in 2007: one state; 2006: seven states; 2005: eight states; 2004: six states; and 2003: four states). Although these reports typically included some LHIs, they did not necessarily include all national LHIs and often reflected a state's own priorities.

Inclusion of a focus on disparities also varied across states. In California, for example, the report included data on 72 HP 2010 objectives and assessed progress on each by sex and race/ethnicity. Most other states reported substantially less detailed disparities data if any.

Back to Top | Article Outline

Conclusions

Federal efforts currently play a major role in supporting states' ability to analyze health disparities, at least on the leading national health indicators. While some states mount their own surveys, most appear to rely heavily on their joint work with the HHS on the NVSS, the BRFSS, the YRBSS, the NIS, and other sources to enhance their ability to measure performance in terms of the goals of HP 2010 at a subnational level.

Through advances made over the years, federal sources now better support subnational estimates than they did previously. Although data availability was a consideration in identifying the LHIs, the fact that state-level estimates can now be constructed for so many indicators is encouraging. However, there are still major gaps. National data systems that support state estimates do a much better job of capturing the data necessary to assess disparities based on some population characteristics (broad categories of race/ethnicity) than others (income, disability, sexual orientation, more specific race/ethnic groups). Even in areas where data capture is not an issue, sample size considerations often greatly reduce the specificity of the estimates that can be supported at the state level. Disparities data for geographical divisions smaller than states often are not reported or cannot be generated by the sources used nationally and within states. These gaps are important because local data may be critical to crafting support for work on disparities and essential to evaluating efforts to target improvements. Our study documents the critical gaps that remain in national collection efforts and highlights the important role of efforts that build on federal and state leadership and financial support to help facilitate the availability of state and local data. The study also documents the wide variation in data collection and analytical capability that exists across states. This variation suggests that states can learn from one another and need support to make better use of the data that are available whatever their limitations.

Continued national leadership is important if current advances in state data collection are to be maintained and strengthened to better address LHI disparities in states and localities. Federal leadership and support is essential to create a minimum “floor” of data available within each state, as well as the assistance and encouragement states may find valuable for pursuing their analysis of disparities. It will be unfortunate if federal budgetary pressures limit such progress. Given that variation in state capacity is likely to persist, continued federal leadership will be essential to the analytical infrastructure necessary for supporting progress in eliminating disparities.

Back to Top | Article Outline

REFERENCES

1. US Department of Health and Human Services. Healthy People 2010. Vols I and II. Washington, DC: US Government Printing Office; 2000.

2. National Committee on Vital and Health Statistics. Eliminating Health Disparities: Strengthening Data on Race, Ethnicity and Primary Language in the United States. Washington, DC: US Department of Health and Human Services; 2005.

3. Institute of Medicine. The Future of Public Health in the 21st Century. Washington, DC: National Academies Press; 2003.

4. Centers for Disease Control and Prevention. CDC Assessment Initiative. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention. http://www.cdc.gov/ncphi/od/ai/ai-bg_new.htm. Accessed June 4, 2008.

5. Bilheimer LT, Sisk JE. Collecting adequate data on race and ethnic disparities in health: the challenges continue. Health Aff. 2008;27(2):383–391.

6. CDC Wonder DATA2010: the Healthy People database. http://wonder.cdc.gov/data2010/focus.htm. Accessed March 22, 2007.

7. Dodd AH, Neuman M, Gold M. Assessment of State Capacity to Identify and Track Disparities in the Leading Health Indicators. Washington, DC: Mathematica Policy Research Inc; 2007.

8. Coffey R, Barret M, Houchens R, Moy E, Andrews R. Methods for applying AHRQ Quality Indicators to Healthcare and Utilization Project (HCUP) data for the Fifth (2007) National Health Care Disparities Report. www.hcup-us.ahrq.gov/reports/methods.jsp. Accessed June 12, 2008.

9. Cervantes IF, Brick JM. CHIS 2005 Methodology Series. Report 1: Sample Design, April 2007. www.chis.ucla.edu/pdf/CHIS2005_method1.pdf. Accessed September 25, 2007.

10. Arias E, Schauman WS, Sorlie P. Race and Hispanic origin reporting on death certificates in the United States: status and effects. Presented at: the 2007 Annual Meeting of the Population Association of America; March 29, 2007; New York, NY.

11. Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System Survey Data. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention. http://apps.nccd.cdc.gov/brfss. Accessed August 15, 2007.

12. Klein RJ, Proctor SE, Boudreault MA, Turczyn KM. Healthy People 2010 Criteria for Data Suppression. Healthy People 2010 Statistical Notes. Atlanta, GA: Centers for Disease Control and Prevention; 2002.

*Although some large localities collect their own data, our focus here was on data provided through federal or statewide efforts. Most localities do not have the resources to collect their own data. Our study aimed to maximize the information we could collect with available resources. Cited Here...

We also reviewed the Environmental Protection Agency's Air Quality Survey. Cited Here...

*Because our focus was on LHIs as a whole and reducing disparities in health, rather than disparities in healthcare, we did not do an extensive review of specialized datasets focused on the latter. We thus may understate recent state work on capturing better data on health coverage at the state and substate levels. Cited Here...

According to Roxanne Andrews of the Agency for Healthcare Research and Quality, in 2005, 8 states that participated in the HCUP had no disparities data reporting and 3 of the remaining 28 could not report four basic codes used (including Hispanics) (Presented at: AcademyHealth's Annual Research Meeting; June 10, 2008). Only 23 states had adequate data to estimate nationwide disparities.8 Cited Here...

*In 2005, disability was characterized in the BRFSS to include those reporting either use of special equipment due to health problems or being limited in activities due to physical, mental, or emotional problems. Cited Here...

Because the YRBSS is conducted in schools, the child's grade level is captured, but not the parental education. Cited Here...

*Privacy constraints and state agreements with the HCUP limit the estimates available from the federal site. The federal HCUP site supports estimates only for urban versus rural areas across states. Cited Here...

Keywords:

health status indicators; Healthy People 2010; state and local health data

© 2008 Lippincott Williams & Wilkins, Inc.

Login

Search for Similar Articles
You may search for similar articles that contain these same keywords or you may modify the keyword list to augment your search.