Click on the links below to access all the ArticlePlus for this article.
Please note that ArticlePlus files may launch a viewer application outside of your web browser.
Rare adverse drug effects may be observed only after a medication has been used widely. Large health care utilization data sets are often the best source of data for analysis of the incidence of such rare adverse effects. A major advantage of health care utilization data is that they reflect routine practice for large and representative populations, compared with the much smaller and often healthier patient populations in clinical trials. Furthermore, analyses of these readily available data sets avoid delays common to primary data collection for such drug risks.
Pharmacoepidemiologic studies using claims data have been criticized for having incomplete information about potential confounders, such as the use of over-the-counter medications (eg, aspirin), body-mass index, or smoking status. Such factors may lead to selective prescribing of drugs, which may result in biased estimates of the association between drugs and adverse health outcomes.1 However, if the distribution of these unmeasured characteristics among drug user categories can be described in a representative subsample, it would be possible to estimate the direction as well as magnitude of potential confounding bias in the main database study.2 Several studies have explored how strong an unmeasured confounder would have to be to explain an elevated relative risk in pharmacoepidemiologic studies.3–5 None of these attempted to quantify the actual magnitude of bias caused by unmeasured confounders.
The relation between selective COX2 inhibitors and acute myocardial infarction (MI) is a recent example of this problem, which has large public health consequences because of the widespread use of these antiinflammatory drugs. Randomized trials of these agents were inconclusive because they limited follow-up to short intervals, excluded patients at high risk for MI, or excluded people taking high dosages of selective COX2 inhibitors.6–11 However, epidemiologic database studies that could overcome some of these limitation have been repeatedly criticized for not being able to control for important clinical confounders that were unmeasured in claims data.
We conducted a large epidemiologic study using Medicare data and computerized pharmacy dispensing data to assess the relationship between selective COX2 inhibitors and myocardial infarction.12 For the analysis presented here, we obtained detailed in-home interview data from a large sample of current Medicare beneficiaries. We assessed the impact of confounding bias by factors not measured in Medicare data, including over-the-counter aspirin use, obesity, smoking, income status, and educational attainment.
To illustrate how we correct effect estimates for unmeasured confounding using external information, we used the potential association between selective COX2 inhibitor use and the incidence of MI. This association was reported by several observational studies using health care utilization databases.12,13 We specifically will apply our results to a recent study in Medicare beneficiaries who had drug coverage through a pharmacy assistance program that had no copay and minimal dispensing fee. Included in the coverage were all selective COX2 inhibitors and nonselective nonsteroidal antiinflamatory drugs (NSAIDs).12 In this study (called the main study) we can adjust for a number of potential confounders (CMeasured in Fig. 1). We used the Medicare Current Beneficiary Survey (MCBS) to estimate the associations between predefined drug exposure categories and selected confounders not measured in Medicare data (CUnmeasured) and estimates of the confounder-disease associations abstracted from the medical literature (Fig. 1). This external information will be used to correct effect estimates in the main study that relied exclusively on claims data. Some independent risk factors for MI were unmeasured in both data sources and thus were not considered, eg, physical activity, cholesterol levels, or blood pressure.
Medicare Current Beneficiary Survey
The MCBS is conducted in a sample of beneficiaries selected each year to be representative of the current Medicare population, including both aged and disabled beneficiaries living in the community or in institutions.14 The MCBS slightly oversamples disabled beneficiaries (under 65 years of age) and the oldest beneficiaries (85 years of age or over).14 Data are obtained from face-to-face interviews by trained interviewers in the beneficiaries' homes or facilities. In the community interview, an effort is made to interview all subjects directly. If a person was unable to answer all questions, a proxy respondent, usually a family member or close acquaintance, was asked to answer the questions. All MCBS participants are interviewed every 4 months and followed for up to 4 years. Each year, a supplemental sample is drawn and persons are added to the MCBS sample to account for growth in the Medicare population and to replenish the sample for surveyed seniors who died, left the survey population after 4 years, or were lost to follow-up. The survey generally has a high response rate (between 85% and 95%) and very high data completeness.15,16 Details of the MCBS sampling plan, interviews, and data items have been previously reported.17
The MCBS sample was drawn from an enrollment list of all persons entitled to Medicare on 1 January 1999, and represented those who were continuously enrolled from 1 January 1999 to 31 December 1999. The total sample size of the 1999 MCBS was 13,106 subjects. For the present analysis, the study population was restricted to persons living in the community (n = 11,984); information on all critical items was 99% complete. We further restricted the study population to MCBS respondents ≥65 years (n = 10,446) who had used at least one medication in 1999 (n = 8,785). The study was approved by the Center for Medicare and Medicaid Services and by the Institutional Review Board of the Brigham and Women's Hospital.
Assessment of Medication Use
At every interview, drug names were recorded by the interviewer from medication bottles of participants in a free-text field of the questionnaire. Raw drug data consisted of prescribed medications and over-the-counter drugs used at least once during 1999. To analyze over-the-counter drug use,18 we obtained raw drug data files and used these files for all analyses, rather then the edited file usually available through Centers for Medicare & Medicaid Services.
MCBS respondents were divided into 3 categories according to their drug use: (1) respondents who used either celecoxib or rofecoxib in 1999 (including those who also used nonselective NSAIDs) classified as selective COX2 inhibitor users; (2) respondents who used only nonselective NSAIDs; (3) respondents who did not use any NSAIDs in 1999 classified as nonusers.
Assessment of Potential Confounders Measured in the MCBS
Potential confounders of interest assessed in the MCBS but unmeasured in Medicare claims data included: BMI (weight in kg/[height in m]2), over-the-counter aspirin use, current smoking, income status, and educational attainment. Body mass index was calculated from the given height and weight information provided in one interview session in 1999. Beneficiaries were categorized as obese if their BMI was higher or equal to 30 kg/m2 according to the WHO definition.19 Any reported use of aspirin (eg, adult aspirin, adult low-dose aspirin, children's aspirin, etc.) was obtained from the raw drug file. Smoking status was dichotomized into current versus former and never. Although there is no formal study on the accuracy of these variables in the MCBS, data quality is presumed good since data were obtained from face-to-face interviews by trained interviewers in the beneficiaries' homes17 and interviewees were asked to bring their medication bottles.
Using the MCBS study population, we estimated the prevalence of exposure, Pr(E), and the prevalence of potential confounders, Pr(C), and the association between exposure and confounder (OREC). We used logistic regression to calculate the corresponding age-and-sex-adjusted OREC, which was used for all subsequent analyses. For initial bias estimates we assumed the null hypothesis that there is no association between selective COX2 inhibitor exposure and the incidence of MI (RRED = 1.0). These bias estimates were later applied to the observed effect estimates that may be different from 1.0. We derived estimates of the confounder-disease associations (RRCD) from the current literature. Literature estimates were derived from large cohort studies or randomized trials after an intensive literature search and expert consultations. If several valid literature estimates were identified, the most extreme (farthest away from the null) was chosen for the base-case analysis, which would lead to more extreme bias estimates.
Based on these estimates and assumptions, a quadratic equation was used to approximate the direction and extent of residual confounding bias that would result from a failure to control for the list of 5 possible confounders (as described in the appendix, available with the electronic version of this article).20 We graphically explored the sensitivity to variations of our base-case literature estimate of RRCD. The joint distribution of unmeasured confounders was not assessed because literature estimates were not available for many confounder combinations. Instead we summed bias estimates over all confounders, weighted by the prevalence of each confounder in the MCBS population. Finally, we calculated the maximum range of bias by summing all negative biases to yield a lower bound estimate and all positive confounders to yield an upper bound. A spreadsheet program is available upon request.
We do not present 95% confidence intervals for the distributions of potential confounders across drug exposure categories, because our sensitivity analysis is based only on point estimates and we did not attempt any inference on differences in the distribution of potential confounders.
Among our study population (n = 8,785) 42% were men. The mean body mass index (± standard deviation) was 25.8 kg/m2 (± 5.3). Seventy percent of study subjects had no college education. Overall, 53% of the population earned $20,000 per year or less. Proxy interviews were used for 12% of subjects.
Table 1 shows distributions of selected characteristics for selective COX2 inhibitor users compared with nonselective NSAID users and nonusers. Across the 3 drug user groups, we observed some differences in the distributions of sex, age, smoking status, and income status of more than 5% points. Obesity was about equally distributed among selective COX2 inhibitor users and nonselective NSAID users as well as current smoking status. Selective COX2 inhibitor users were more likely than nonselective NSAID users to have higher income status (52% versus 44%). The prevalence of aspirin use was about equal among all 3 drug user categories. The distributions of educational attainment slightly differed in the 3 drug groups, with selective COX2 inhibitor users being more likely than nonselective NSAID users to have a college or higher degree.
Users of celecoxib and rofecoxib, the 2 selective COX2 inhibitors on the market in 1999 were similar with respect to over-the-counter aspirin use, obesity, smoking, income status, or educational attainment (less than 5% difference, Table 1). Selective COX2 inhibitor users were less likely to be smokers and tended to have higher income status than ibuprofen users.
Literature estimates of the associations between possible confounders and the incidence of MI are shown in Table 2. Relative risks ranged from 0.7 for aspirin use to 3.1 for smoking status.
We calculated residual confounding bias in claims data risk estimates of a potential association between selective COX2 inhibitors and the incidence of MI separately for each of the 5 potential confounders (Table 3). Confounding was quantified as the ratio of the apparent RRED over the assumed true RRED of 1.0 and as percent bias of the observed claims data estimate. We found that failure to adjust for each of 5 potential confounders would modestly underestimate the association between selective COX2 inhibitor use and MI. Uncontrolled low educational attainment status was the single strongest confounder, causing a bias of −2.4% when comparing selective COX2 inhibitor versus nonselective NSAIDs. The net confounding bias, expressed as the sum of all component biases weighted by the population prevalence of each confounder, was −1.6% (Table 4). The range from the most extreme negative bias to the most extreme positive bias was −5.9% to +0.3%.
Unmeasured confounding bias was similar when comparing COX2 users versus nonusers or naproxen users (Table 4). The strongest negative bias was observed when comparing rofecoxib with naproxen use (−3.2%; −9.2% to 0%). Over-the-counter aspirin use appeared to cause no bias for either comparison.
Bias estimates for obesity and income status were fairly sensitive to changes in the estimate of the confounder-MI association derived from the literature (Fig. 2), whereas other confounders (smoking, aspirin use, and education) were less sensitive. This sensitivity was primarily due to stronger exposure-confounder associations, for example the OREC for obesity in Figure 2A was 1.55 and 1.26 for the upper 2 curves but 0.99 and 1.00 for the flat lines, indicating 0% bias. The low prevalence of current smoking is mainly responsible for the low sensitivity for this variable despite the moderately strong exposure-confounder associations (smallest OREC = 0.84).
The VIGOR trial6 raised the possibility that selective COX2 inhibitors may be associated with MI. Observational studies from health care utilization data12,13 may offer more generalizable evidence of such an association but these studies were criticized for their potential of unmeasured confounding. Our findings indicate that the omission of 5 potential confounders that are independently associated with MI was not likely to change the interpretation of risk estimates based on health care utilization data in studying the association between selective COX2 inhibitors and myocardial infarction in the elderly. All confounding factors examined here would moderately bias the comparison of selective COX2 inhibitors with nonselective NSAIDs towards the null. As a result, analyses without data on these confounders would be expected to produce slightly conservative relative risk estimates. These findings are robust towards variations in literature estimates of the underlying confounder-MI associations.
In a previous study, a telephone survey of 3,500 United Healthcare members age 64 years and younger showed that selective COX2 inhibitor users reported about the same prevalence of use of aspirin and tobacco as nonselective NSAID users.21 The authors concluded that a relative risk estimate for the association between selective COX2 inhibitor use and myocardial infarction would change by no more than 5–6% if aspirin and smoking status were observable and adjusted in a claims data analysis. While these results were reassuring, it was unclear whether they can be generalized to the much larger population of elderly users and to other potential confounders.
Our findings can be applied to 2 recent epidemiologic studies on the association between selective COX2 inhibitor use and the incidence of MI. A cohort study of 380,000 Medicaid beneficiaries by Ray et al13 reported a relative risk of 1.70 (95% CI = 0.98–2.95) in high dose rofecoxib users (>25mg) compared with nonusers. A nested case-control study of beneficiaries of 2 state-funded pharmacy assistance programs for the elderly (age ≥65years) by Solomon et al12 reported a relative risk of 1.58 (1.04–2.40) in high dose rofecoxib users (>25mg) compared with nonusers. The same study found a relative risk of 1.68 (1.00–2.84) comparing high dose rofecoxib to high-dose naproxen use.
Unfortunately, there are no randomized trial data that could be used for a direct validation of our bias assessment. The VIGOR study6 is the only randomized trial to date that allows a limited comparison of findings. This randomized clinical trial of 8,076 patients with rheumatoid arthritis found that individuals who took high-dose rofecoxib (50mg/d) had a 5-fold increase in risk of MI compared with those who took 1000 mg/d naproxen (95% CI = 1.4–10).6 If this difference in MI risk between rofecoxib and naproxen were generalizable to the populations in observational studies, this finding would suggest that observational studies would substantially underestimate the effect found in a randomized trial even after correcting for unmeasured confounding. Although a small proportion of the difference in the observed effects can be explained by unmeasured confounding, there were some important differences between the patients in VIGOR and in the observational studies that may not allow a fair comparison: The observational study using Medicare data12 was restricted to elderly patients ≥ 65 years, whereas the VIGOR trial was limited to patients with rheumatoid arthritis not taking aspirin; medication dosages in the observational studies12,13 were >25mg/d rather than 50 mg/d in the VIGOR study; and some patients classified as nonusers in the observational studies may have been exposed to free samples.22
Our approach to assessing the direction and magnitude of unmeasured confounding makes several simplifying assumptions. Exposure, confounder, and outcome were all coded as dichotomous variables. While this may not be of concern for the outcome of interest (incidence of MI), for drug exposure, or for some potential confounders (including sex, smoking, and education), it may over-simplify the relation between other confounders and outcomes. Choosing alternative cutpoints in confounder variables such as BMI may change the strength of an association. Because the confounder-MI associations in our example are likely to be monotonic, dichotomized confounder variables would not change the direction of the bias estimates.23 We further assumed that the unobserved true exposure-disease association was 1.0 when we assessed the magnitude of bias. If the true association is different from 1.0 our estimate of bias may be slightly inconsistent. However, the closer an association is to the null the less our bias estimate will diverge from the true bias. Lastly, we did not consider the joint distribution of unmeasured confounders. Instead we summed bias estimates over all confounders, weighted by the prevalence of each confounder in the MCBS population as an approximation of the net bias produced by 5 unmeasured confounders. Although the extremes of this range are unlikely, their use would lead to the most conservative interpretation of the data.
We further made the simplifying assumption that the unmeasured confounders are independent of the measured confounders conditional on exposure status.24 If measured and unmeasured confounders are associated this can lead to an overestimation of the magnitude of bias.
Valid bias assessment depends on the interviews being performed in a representative sample of the main claims data study. Given that the MCBS was designed to be representative for Medicare beneficiaries and that it had a high response rate and high degree of data completeness, this data source appears to be useful and readily available for bias assessment. Although we could identify 5 important risk factors for MI in the MCBS survey, information on other independent risk factors (eg, physical activity, cholesterol levels, or blood pressure) was not available, which may be a source of residual confounding beyond our assessment of bias.
The absence of information on potential confounders is a common criticism of observational studies based on health care utilization data. Our analysis demonstrates that patient survey data from the MCBS can be used to quantitatively assess confounding bias in pharmacoepidemiologic studies using Medicare claims data.
1. Walker AM. Confounding by indication. Epidemiology
2. Gail MH, Wacholder S, Lubin JH. Indirect corrections for confounding under multiplicative and additive risk models. Am J Ind Med
3. Walker AM. Newer oral contraceptives and the risk of venous thromboembolism. Contraception
4. Wang PS, Bohn RL, Glynn RJ, Mogun H, Avorn J. Zolpidem use and hip fractures in older people. J Am Geriatr Soc
5. Psaty BM, Koepsell TD, Lin D, Weiss NS, et al. Assessment and control for confounding by indication in observational studies. J Am Geriatr Soc
6. Bombardier C, Laine L, Reicin A, et al. Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med
7. White WB, Faich G, Whelton A, et al. Comparison of thromboembolic events in patients treated with celecoxib, a cyclooxygenase-2 specific inhibitor, versus ibuprofen or diclofenac. Am J Cardiol
8. Mukherjee D, Nissen SE, Topol EJ. Risk of cardiovascular events associated with selective COX-2 inhibitors. JAMA
9. Konstam MA, Weir MR, Reicin A, et al. Cardiovascular thrombotic events in controlled, clinical trials of rofecoxib. Circulation
10. Reicin A, Shapiro D, Sperling RS, Barr E, Qinfen Y. Comparison of cardiovascular thrombotic events in patients with osteoarthritis treated with rofecoxib versus nonselective nonsteroidal anti-inflammatory drugs (ibuprofen, diclofenac, and nabumetone). Am J Cardiol
11. White WB, Faich G, Whelton A, et al. Comparison of thromboembolic events in patients treated with celecoxib, a cyclooxygenase-2 specific inhibitor, versus ibuprofen or diclofenac. Am J Cardiol
12. Solomon DH, Schneeweiss S, Glynn RJ, Kiyota Y, Levin R, Avorn J. The relationship between selective COX-2 inhibitors and acute myocardial infarction. Circulation
13. Ray WA, Stein CM, Daugherty JR, Hall K, Arbogast PG, Griffin MR. COX2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet
14. Eppig FJ, Chulis GS. Matching MCBS and Medicare data: The best of both worlds. Health Care Financing Review
15. Adler GS. A profile of the Medicare Current Beneficiary Survey. Health Care Financing Review
16. Adler GS. Medicare beneficiaries rate their medical care. Health Care Financing Review
18. Davis M, Poisal J, Chulis G, Zarabozo C, Cooper B. Prescription drug coverage, utilization and spending among Medicare beneficiaries. Health Affairs
19. WHO. Expert committee physical status: the use and interpretation of anthropometry. WHO technical report series N0. 854;452, Geneva, 1990.
20. Walker AM. Observation and Inference. Epidemiology Resources Inc., Newton, 1991, 120–124.
21. Velentgas P, Cali C, Diedrick G, et al. A survey of aspirin use, non-prescription NSAID use, and cigarette smoking among users and non-users of prescription NSAIDs: estimates of the effect of unmeasured confounding by these factors on studies of NSAID use and risk of myocardial infarction. Pharmacoepidemiol Drug Safety
22. Jacobus S, Schneeweiss S, Chan KA. Exposure misclassification as a result of free sample drug utilization in automated claims databases and its effect on pharmacoepidemiologic studies of selective COX-2 inhibitors. Pharmacoepidemiol Drug Safety, published online: 29 Jul 2004 DOI: 10. 1002/pds. 981.
23. Rothman K, Greenland S. Modern Epidemiology, 2nd
ed. Philadelphia, PA: Lippincott; 1998:205.
24. Hernan MA, Robins JM. Letter to the editor: Assessing the sensitivity of regression results to unmeasured confounders in observational studies. Biometrics
25. Reuterwall C, Hallqvist J, Ahlbom A, et al. Higher relative, but lower absolute risks of myocardial infarction in women than in men: Analysis of some major risk factors in the SHEEP study. J Intern Med
26. Hu FB, Rimm EB, Stampfer MJ, Ascherio A, Spiegelman D, Willet WC. Prospective study of major dietary patterns and risk of coronary heart disease in men. Am J Clin Nutr
27. Wilson PWF, D'Agostino RB, Sullivan L, Parise H, Kannel WB. Owerweight and obesity as determinants of cardiovascular risk. The Framingham experience. Arch Intern Med
28. Manson JE, Stampfer MJ, Colditz GA, et al. A prospective study of aspirin use and primary prevention of cardiovascular disease in women. JAMA
29. Cook NR, Hebert PR, Manson JE, et al. Self-selected posttrial aspirin use and subsequent cardiovascular disease and mortality in the Physician's Health Study. Arch Intern Med
30. The SALT collaborative group. Swedish aspirin low-dose trial (SALT) of 75mg aspirin as secondary prophylaxis after cerebrovascular ischaemic events. Lancet
31. Psaty BM, Furberg CD, Kuller LH. Traditional risk factors and subclinical disease measures as predictors of first myocardial infarction in older adults. Arch Intern Med
32. Freund KM, Belanger AJ, D'Agostino RB, et al. The health risks of smoking. The Framingham Study: 34 years of follow-up. Ann Epidemiol
33. Willett WC, Green A, Stampfer MJ, et al. Relative and absolute excess risk of coronary heart disease among women who smoke cigarettes. N Engl J Med
34. Bobak M, Hertzman C, Skodova Z, Marmot M. Own education, current conditions, parental material circumstances, and risk of myocardial infarction in a former communist country. J Epidemiol Community Health
35. Pocock SJ, Shaper AG, Cook DG, Phillips AN, Walker M. Social class differences in ischaemic heart disease in British men. Lancet
. 1987;July 25:197–201.
36. Lynch JW, Kaplan GA, Cohen RD, Tuomilehto J, Salonen JT. Do cardiovascular risk factors explain the relation between socioeconomic status, risk of all-cause mortality, cardiovascular mortality and acute myocardial infarction? Am J Epidemiol
Supplemental Digital Content
© 2005 Lippincott Williams & Wilkins, Inc.