Health outcomes are powerfully affected by socioeconomic status (SES) in ways that we still cannot explain. One of the great enigmas of modern day epidemiology is the observation found throughout the world that mortality rates vary enormously across socioeconomic levels even when known individual risk factors are taken into account. 1–6 This puzzle has led to substantial concern that efforts to assess the performance of health care providers by profiling outcomes may be biased if the SES of the patients they treat is not taken into account. 7–9 For a variety of reasons, providers and insurers have not historically collected data on the income and education of their patients and do not systematically collect data on occupation. It seems likely that it would be difficult for them to do so, at least in this country, given concerns that they may use these data to select low-cost populations to care for and given general societal concerns about privacy. 10
See p 8
In the article in this issue by Fiscella et al, 11 the authors ask whether the addresses of individuals can be used to assign an average education level taken from census data as a proxy for individual-level education, for use in adjusting provider profiles. Home address is available for almost all patients cared for by a provider, and its use avoids the collection and storage of additional information that might be considered sensitive by many. The authors conclude that education level assigned by zip code of residence has an effect on several measures of provider performance similar to that found with individual education level variables.
Fiscella and colleagues should be commended for asking important questions about the role of patient socioeconomic factors in efforts to assess quality of care in this and several related articles. 7,11,12 There are 2 related inferences that one could draw from this work: first, that zip code data can be used as a proxy for individual-level education measures in analyses with health outcomes as the dependent variable, and second, that provider profiles should be adjusted for SES using the (almost) universally available zip code–level education measures.
It is important to think about the potential differences between a geographic and an individual-level measure that have the same name (education in this case). Most obviously, the geographic measure can be thought of as an imperfectly measured individual variable. As such, the variable could be used to adjust for the different SES compositions of groups of people in an analysis of some health outcome. A second and increasingly popular way to think about the geographically based measure is that it captures some aspect of the context in which the individuals live. 13,14 In other words, attributes of a geographic area with very high average education level, such as school quality or social services, may contribute to an outcome independently of relationships of individual education levels to the outcome. The search for these contextual effects has become very fashionable, although they often prove to be elusive. 15
Can we predict the potential problems of using aggregate SES as a measure of individual SES? Although a large body of literature in economics discusses the statistical issues of the “aggregation problem,” the authors of these articles are principally concerned with making inferences about individuals using analyses with aggregate variables as both dependent and independent variables. 16–18 I am aware of only one article, by Geronimus et al, 19 that presents a theoretical discussion, based on an explicit statistical model, of the impact of using aggregate SES to study individual health outcomes. They conclude that the coefficient of the individual-level SES variable is downwardly biased by the imperfect measurement of SES and upwardly biased (in most cases) by the contextual effects. More to the point, the addition of an aggregate SES variable will partially, but not completely, remove SES confounding of the coefficient of a third variable (eg, a measure of physician quality). If there are SES contextual effects, this will add an additional bias to the coefficient of a third variable.
This description of multiple sources of bias with potentially positive or negative direction sounds bad, but it is important to think about what our goal is. Do we want to adjust outcomes for only the compositional or both the compositional and contextual relationships between SES and outcome? If the answer is the latter, then we would accept the so-called contextual bias as part of what we want to include when adjusting profiles, and the only downside of using the aggregate variable would be that the compositional effects would be partially but not completely controlled. Is the glass half full or half empty?
A simpler empirical approach is to try to validate the use of geographic SES variables by demonstrating in an example that similar conclusions are drawn when aggregate or individual SES variables are used, and then infer that this is likely to be generally true. This is the approach that Fiscella et al 11 take in their study, as have others 20 (including myself 21). Fiscella et al show in their article that physician profiles adjusted by data from the 2 sources are highly correlated and that the addition of individual-level education data to a model with census-derived education explains only marginally more of the physician-level variation. 11
Unfortunately, there is no simple way to summarize how much rankings change when adjusted by different measures of SES. Their measure of mean absolute rank change or correlation over the entire rank list may not give the most important information if physician rankings only have an impact in the real world on those physicians who are in the lowest or highest 10th percentiles. In addition, as the authors point out, it is difficult to formally test the significance of a change in rankings. It is possible, using simulation techniques, to estimate confidence intervals for physician rankings in health outcome profiles, and they do tend to be very wide, 22 which suggests that there is even more noise in rankings based on profiles than in the profile measures themselves.
Thus, although I find their data to support a hypothesis of equivalence, I retain a bit of uncertainty over how big a difference in rankings is important and how well their measures would pick up this difference. The other problem with using empirical examples without a theoretical statistical model is that it is more difficult to justify generalizations to other populations and outcome variables. 23
Fiscella et al 11 do not explicitly examine the potential role of the geographic contextual effects of SES on health outcomes, and indeed, to do so requires both individual and census measures in the same model and the explicit definition of patients as nested both within physicians and overlapping geographic areas. Such a model is technically feasible but complex. 24 Should we be looking for these geographic contextual effects?
The answer to this question really depends on the degree to which we can develop a compelling conceptual model in which these contextual effects play an important role. Fiscella et al, 11 perhaps wisely, limit themselves to assessing how well census-derived SES can act as an imperfectly measured proxy for individual-level SES.
We should also be sure to take a step back and ask whether physicians are likely to have an effect on cross-sectional measurements of patient satisfaction, physical health status, and mental health status. Fiscella et al 11 do not tell us how much variation there is at the physician level, but based on other work, we would expect it to be small. 9,25 Is there likely to be a bigger difference between physician rankings produced using individual and census-derived SES measures if we profile physicians using measures on which they have a larger impact? Of course, if the amount of physician-level variation is negligible, then we shouldn’t care about the rankings at all, and the question of whether they are different after controlling for different measures of SES is moot.
Should we now recommend that all physician profiles be adjusted using the average educational attainment of each patient’s home zip code? To make this inference from the study by Fiscella et al 11 requires most critically our acceptance of the assumption stated explicitly in their introduction that patient SES is unrelated to physician quality. Obviously, if it turns out that lower-SES patients are taken care of by lower-quality physicians, we will not want to adjust physician profiles by any measure of SES, be it individual or geographic, because we will lose our ability to find differences in provider performance. I am not as sanguine as Fiscella et al, who believe that prior research has convincingly shown an absence of relationship between quality of care and patient SES attributes. However, given the known relationship between SES and health outcomes, I think it is reasonable to assume the independence of SES and quality in profiling while keeping it as an open research question.
So where does that leave us? Clearly, further work could be done to examine the implications of the underlying statistical models. This can be done best after estimating the magnitude and direction of the compositional and contextual effects in some of these data sets that contain both individual and geographic measurements of SES. I personally do not believe that physician-level profiling is justified unless there is some evidence of an important amount of variation at the physician level and that physician-level measures are reliable, given the amount of variation and the number of patients in a physician’s panel. 9 However, assuming that these conditions have been satisfied, I think that it would be reasonable to include census-based measures of education in provider profiles of health outcomes while maintaining the view that the use of both individual and geographic SES measurements is optimal.
1. University of Edinburgh, Research Unit in Health and Behavioural Change. Changing the Public Health. Chichester, NY: Wiley, 1989.
2. Geronimus AT, Bound J, Waidmann TA. Poverty, time, and place: Variation in excess mortality across selected US populations, 1980–1990. J Epidemiol Commun Health 1999; 53: 325–334.
3. Fiscella K. Is lower income associated with greater biopsychosocial morbidity? Implications for physicians working with underserved patients. J Fam Pract 1999; 48: 372–377.
4. Lantz PM, House JS, Lepkowski JM, et al. Socioeconomic factors, health behaviors, and mortality: Results from a nationally representative prospective study of US adults. JAMA 1998; 279: 1703–1708.
5. House JS, Kessler RC, Herzog AR. Age, socioeconomic status
, and health. Milbank Q 1990; 68: 383–411.
6. Kaplan GA, Keil JE. Socioeconomic factors and cardiovascular disease: A review of the literature. Circulation 1993; 88 (pt 1):1973–1998.
7. Fiscella K, Franks P. Influence of patient education on profiles of physician practices. Ann Intern Med 1999; 131: 745–751.
8. Zaslavsky AM, Hochheimer JN, Schneider EC, et al. Impact of sociodemographic case mix on the HEDIS measures of health plan quality. Med Care 2000; 38: 981–992.
9. Hofer TP, Hayward RA, Greenfield S, et al. The unreliability of individual physician “report cards” for assessing the costs and quality of care of a chronic disease [see comments]. JAMA 1999; 281: 2098–2105.
10. Kircheimer B. Public fears enter into equation. Mod Healthc October 2000;supplement(Eye on Info):42.
11. Fiscella K, Franks P. Impact of patient socioeconomic status
on physician profiles: A comparison of census-derived and individual measures. Med Care 2001; 39: 8–14.
12. Fiscella K, Franks P, Gold MR, et al. Inequality in quality: Addressing socioeconomic, racial, and ethnic disparities in health care. JAMA 2000; 283: 2579–2584.
13. Kaplan GA. People and places: Contrasting perspectives on the association between social class and health. Int J Health Serv 1996; 26: 507–519.
14. Robert SA. Community-level socioeconomic status
effects on adult health. J Health Soc Behav 1998; 39: 18–37.
15. Duncan C, Jones K, Moon G. Context, composition and heterogeneity: Using multilevel models in health research. Soc Sci Med 1998; 46: 97–117.
16. Cleave N, Brown PJ, Payne CD. Evaluation of methods for ecological inference. J R Stat Soc Series A (Stat Soc) 1995; 158: 55–72.
17. Firebaugh G. A rule for inferring individual-level relationships from aggregate data. Am Sociol Rev 1978; 43: 557–572.
18. Moulton BR. An illustration of a pitfall in estimating the effects of aggregate variables on micro units. Rev Econ Stat 1990; 72: 334–338.
19. Geronimus AT, Bound J, Niedert LJ. On the validity of using census geocode characteristics to proxy individual socioeconomic characteristics. J Am Stat Assoc 1996; 91: 529–537.
20. Krieger N. Women and social class: A methodological study comparing individual, household, and census measures as predictors of black/white differences in reproductive history. J Epidemiol Commun Health 1991; 45: 35–42.
21. Hofer TP, Wolfe RA, Tedeschi PJ, et al. Use of community versus individual socioeconomic data in predicting variation in hospital use. Health Serv Res 1998; 33 (2 pt 1):243–259.
22. Goldstein H, Spiegelhalter DJ. League tables and their limitations: Statistical issues in comparisons of institutional performance. J R Stat Soc Series A 1996; 159: 385–443.
23. Krieger N, Gordon D. Re: “Use of census-based aggregate variables to proxy for socioeconomic group: evidence from national samples.” Am J Epidemiol 1999;150:892–896. Letter.
24. Goldstein H. Multilevel Statistical Models. 2nd ed. New York, NY: Halstead Press, 1995.
25. Sixma HJ, Spreeuwenberg PM, van der Pasch MA. Patient satisfaction with the general practitioner: A two-level analysis. Med Care 1998; 36: 212–229.