Modeling the 4D Study

Statins and Cardiovascular Outcomes in Long-Term Hemodialysis Patients with Diabetes

Abstract

Background and objectives: Randomized, controlled trials (RCTs) are the gold standard for defining causal inferences but are sometimes not feasible because of cost, ethical, or time considerations. We explored the accuracy and potential use of a “simulated trial” through the modeling of a previously published RCT, Die Deutsche Diabetes Dialyse Studie (4D Study), a landmark study that investigated the cardiovascular benefit of atorvastatin use in 1255 patients with ESRD.

Design, setting, participants, & measurements: Using a large historical database of interventions and outcomes in dialysis patients, we conducted an observational model of the 4D Study in dialysis patients who had type 2 diabetes and were prescribed a statin (5144 patients) and matched to a non–statin user (5144 control subjects) before multivariate modeling. Inclusion, exclusion, and outcome parameters of the study, as prespecified by the 4D Study, were strictly modeled in this analysis.

Results: In covariate- and propensity-adjusted Cox regression, statin use (versus nonuse) was associated with a decrease in the composite primary outcome of cardiac death, nonfatal myocardial infarction, and stroke. Statin use was also associated with a decrease in cardiovascular mortality and all cardiac events combined. The hazard ratios in this observational model were numerically comparable to the hazard ratios reported in the 4D Study; however, because of the larger number of patients “enrolled,” results in this simulated study achieved statistical significance.

Conclusions: Statin use was associated with some cardiovascular benefit in a simulated trial of patients with ESRD; however, the size of benefit was considerably smaller than that seen in the general population. Such simulated trials may represent an exploratory, cost-effective option when RCTs are not immediately feasible.

Evidence-based medicine has promulgated a hierarchy of best research evidence in which randomized, controlled trials (RCTs) remain the gold standard for determining standards of care (^1,2) and causal inference. Because RCTs minimize potential confounding (³), such studies have become a requirement for the approval of new therapies by national regulatory bodies such as the US Food and Drug Administration.

In the ESRD population, very few RCTs have been successfully carried out, mainly because of the prohibitive costs of conducting a well-designed RCT (⁴). When cost, time, or ethical factors prevent the blind randomization of participants, a well-done epidemiologic study may represent a reasonable alternative for hypothesis testing and generation (⁵). Such analytic approaches have become increasingly prevalent in the dialysis research community with the organization of clinical databases through US Renal Data System (⁶), Dialysis Outcomes and Practice Patterns Study (DOPPS) (⁷), and large dialysis providers. Although valuable, it remains uncertain how reliable a rigorous ESRD observational study performs relative to an RCT, especially with the evolution of newer design strategies that minimize the effect of confounding (⁸).

Die Deutsche Diabetes Dialyze Studie (4D Study) (⁹) was a landmark RCT of the ESRD population that examined the impact of atorvastatin on cardiovascular outcomes. We modeled the 4D Study using clinical data abstracted from a large, national population of patients with ESRD. Our observational model strictly adhered to the same criteria and time period of the 4D Study (¹⁰), such that point estimates between the prospective trial and its observational simulated model could be implicitly compared to assess residual bias.

Materials and Methods

Database Description

Fresenius Medical Care North America is a large provider of long-term renal replacement therapy for >115,000 patients with ESRD in the United States. All clinics and ancillary services are networked through an information system that prospectively collects demographic, medication, comorbidity, outcome, laboratory, process, and patient encounter events for the purpose of health care delivery. Deidentified clinical data sets can be retrospectively extracted from the system for research purposes, as further described in previous journal publications (^11–15). Together, the setup and process provide the three pillars of a good surveillance system: Data validity, informative results, and feasible implementation (^16,17).

Study Design and Cohort

To re-create the 4D Study, we strictly adhered to the same eligibility criteria for the 4D Study as outlined in a previous publication (¹⁰) (Table 1). We anticipated the same 8% reduction in the incidence of the primary end point, as reported by the 4D Study, in our power calculations (β = 0.90, α = 0.05), which estimated a sample size of >8000 patients. To facilitate the attainment of this target, we placed no restrictions on vintage and extended the recruitment period to encompass January 1, 1997, through December 31, 2003, during which time we “enrolled” all new users of hepatic hydroxymethyl glutaryl–CoA (HMG-CoA) reductase inhibitors (users) among a national population of patients who had type 2 diabetes and were receiving long-term hemodialysis (49% of population). To decrease the effect of change in practice patterns, patient outcomes were followed only to October 31, 2004, which coincides with the date the 4D Study was presented at the American Society of Nephrology Renal Week (¹⁸). All statin users and nonusers were required to have at least 90 days of dialysis therapy in a participating dialysis unit without any previous statin prescription.

Table 1: Comparison of study design between this study and the 4D Study

Potential nonusers were also patients who had type 2 diabetes but no previous statin prescription on the index date (the date a statin was started for a statin user). For each user, a nonuser within the same facility was randomly matched on the most recent total cholesterol level and vintage to form a study cohort for subsequent propensity and multivariate (46 parameters) modeling. Cholesterol levels that were drawn before the initiation of statin therapy were matched to similar untreated lipid profiles in non–statin users. Matching was done using a modified “greedy macro” (^19,20) that was structured so that each patient could be enrolled only once in the analysis between the study period of January 1, 1997, through December 31, 2003. We were unable to match on LDL levels because only 6.4% of the study cohort had received LDL testing (measured or calculated), whereas 80% of patients had a total cholesterol level recorded.

Exclusion criteria paralleled those used in the 4D Study (¹⁰) with the exception that patients with any LDL level or dialysis vintage were eligible for the study. Exclusion for “prior sensitivity to HMG-CoA reductase inhibitors” was defined as any discontinuation of statin use before study enrollment. The study was reviewed by the Fresenius Medical Care North America Office of Compliance and designated exempt research under 45 CFR 46.101 (b).

End Points and Covariates

The primary end point was a composite of death from cardiac causes, fatal stroke, nonfatal myocardial infarction (MI), and nonfatal stroke. Secondary end points included all-cause mortality, all cardiac events combined, all cerebrovascular events combined, and the absolute change in albumin 12 months after statin initiation. Death, stroke, and cardiovascular outcomes were classified on the basis of physician-written hospitalization discharge summaries and death records charted in the electronic medical records. Forty-six baseline covariates (listed in Table 3) were defined as the most recent value on the date of study enrollment (i.e., measured before a statin was started). Fasting blood glucose was used to adjust for the degree of diabetic control and disease severity, because only 65.7% of patients had glycosylated hemoglobin readings at baseline.

Table 2: Baseline patient characteristics

Table 3: Study outcomes by statin exposure

Statistical Analysis

Baseline characteristics of the study patients by statin use are reported in parallel with the characteristics of the 4D Study cohort (Table 2). Covariate (46 parameters) adjusted Cox regression models were implemented using forward variable selection with variable exit criteria set at P < 0.20 to determine the relative hazard of statin use versus nonuse. Patients were right-censored at the time they left the facility (because of discharge, death, transfer, withdrawal, transplantation, or attainment of outcome) or at the change of their statin status. Comparisons between the hazard ratios (HRs) of this study and the 4D Study were done using t tests. For validation purposes, we also performed an additional intention-to-treat (ITT) analysis in which patients were not censored if their statin status changed after enrollment in the study. To balance for potential confounding by indication, all Cox models were weighted (^21,22) according to their reciprocal probability of receiving statin therapy (propensity score [²³]) in the users, and 1 − probability of receiving statin therapy in the nonusers. Logistic models were used to calculate a subject's probability of receiving therapy as a function of the baseline parameters as explained further in Table 2.

We also determined the relationship between the primary outcome with the daily statin dosage (mg) in patients who remained enrolled in the analysis at 90 days, the time point when statin dosage was ascertained. We assumed that 90 days represented a reasonable period for a physician to titrate the statin dosage to a steady-state level, after which the analysis became ITT. All statin dosages were converted to a uniform value on the basis of their published efficacy to reduce LDL (^24–26). Thus, 20 mg of atorvastatin was regarded as the equivalent to 80 mg of simvastatin, because both reduce LDL by approximately 47%. Generalized propensity scores by quintiles of statin dosage were calculated using multinomial logistic regression and incorporated as inverse weights into the Cox models to adjust for potential confounding by indication (²⁷).

The HR for the primary outcome by statin type was also calculated through the inclusion of three separate categorical variables in the Cox model (atorvastatin versus untreated; simvastatin versus untreated; other statin versus untreated), with a generalized propensity scoring method. Statistical difference between the type of statin (atorvastatin versus simvastatin versus other statin) was determined using a Wald test for joint hypothesis testing between multiple regression coefficients.

Subgroup analyses by baseline vintage, total cholesterol, and comorbidities were also performed. Because increased baseline medication use was associated with statin prescription (Table 2), we also stratified by angiotensin-converting enzyme inhibitor, angiotensin receptor blocker, β blocker, and antiplatelet use to determine whether concurrent cardioprotective medications confounded the effect statin therapy on the primary outcome. Effect modification testing between statin use and covariates was done through the addition of interaction terms to the Cox model. All statistical analysis was done using SAS 9.1 (SAS Institute, Cary, NC).

Results

Patients

A total of 5144 new statin users were randomly matched to an equivalent nonuser (as shown in the schematic pathways in Figure 1). As expected, several baseline differences between the study groups were noted. These measured baseline differences became insignificant after balancing with propensity adjustment (Table 2). Among statin users, the average equivalent atorvastatin dosage was 33 mg/d. The mean length of follow-up was 1.6 years in the statin user group and 1.8 years in the non–statin user group. Under an ITT assumption whereby patients were not censored if their statin status deviated from their initial assignment, the mean length of follow-up was 2.1 years in the statin user group and 2.0 years in the non–statin user group.

Figure 1: Schematic pathway of the study.

Lipid Levels

A total of 16% of patients had a baseline cholesterol level of >200 mg/dl, which is above the level recommended by the National Cholesterol Education Program (²⁸). The difference in baseline total cholesterol (i.e., cholesterol level before statin treatment initiation) between the two groups was small (absolute difference of 5 mg/dl as shown in Table 2), suggesting that users and nonusers were well matched at the start. During follow-up, statins effectively exerted a lipid-lowering effect (Figure 2) on the study cohort. Patients who were on statin therapy for 12 months experienced a 17.6-mg/dl decrease in total cholesterol, whereas a 9.4-mg/dl decrease was seen in non–statin users (P < 0.0001). Both groups also demonstrated a continuous decrease in lipid levels over time, which is consistent with the 4D Study findings and results reported by another study (²⁹).

Figure 2: Absolute change in cholesterol by statin status. Data are means (SE) at years after enrollment. Total cholesterol levels decreased by 17.6 mg/dl after 1 year of statin therapy, whereas non–statin users saw only a 9.4-mg/dl decrease during the same period (P < 0.0001).

Primary Outcomes

Patients who were on statins had a decreased composite incidence of cardiac death, MI, fatal stroke, and nonfatal stroke of 14.8 events per patient-year, whereas the incidence in non–statin users was found to be 15.6 events per patient-year. Covariate- and propensity-adjusted (c-statistic for propensity score = 0.74) models demonstrated a treatment benefit of 10% (P = 0.01) on the primary outcome (Table 3). The beneficial effects of statin therapy were seen across all cardiovascular outcomes, whereas the rate of nonfatal stroke was 25% (P = 0.03) higher in patients who were prescribed statins when compared with non–statin users. These findings, for the most part, were numerically comparable to HRs reported in the 4D Study (Table 4).

Table 4: Comparison of study outcomes with outcomes of 4D Study

The results remained consistent with the ITT analysis (Table 3). Each 20-mg increase in atorvastatin dosage (versus no atorvastatin) was associated with a 5% (P = 0.01) decrease in the incidence of the primary outcome. High-dosage statin (192 patients with ≥80 mg/d atorvastatin matched to 192 non–statin users) therapy was not associated with an increased benefit (HR for primary outcome = 1.19; 95% confidence interval [CI] 0.77 to 1.86). No statistical superiority among statin types could be demonstrated on the primary outcome (atorvastatin versus simvastatin versus other statins, P = 0.24).

Secondary Outcomes

All-cause mortality was 18% lower (P < 0.0001) in statin users when compared with nonusers (Table 3). Patients who were on statins had a significant 10% decreased risk for any cardiovascular event (P = 0.02) and a nonsignificant 15% increased risk for any cerebrovascular event (P = 0.10), which again was similar to the findings of the 4D Study (Table 4). A small but significant increase in albumin concentration was noted in statin users when compared with nonusers (Figure 3). At 1 year, patients who were on statins (versus patients who were not on statins) were associated with a 0.03-g/dl greater increase in albumin (P = 0.01) when compared with their baseline level (i.e., albumin measured before the initiation of a statin).

Figure 3: Absolute change in albumin by statin status. Data are means (SE) at years after enrollment. Albumin levels increased by 0.06 g/dl after 1 year of statin therapy, whereas non–statin users saw only a 0.03-g/dl increase during the same period (P = 0.02).

Stratified Models

Stratified models demonstrated a relative risk reduction for the composite incidence of cardiac death, MI, fatal stroke, and nonfatal stroke with statin use when compared with nonuse in 42 of 43 strata (Figure 4). The beneficial effect of statin use (versus nonuse) was further magnified in patients with higher total cholesterol levels (Figure 4) and in those with preexisting vascular disease (i.e., secondary prevention): MI (HR 0.62; 95% CI 0.43 to 0.89), coronary artery disease (HR 0.75; 95% CI 0.62 to 0.89), peripheral vascular disease (HR 0.76; 95% CI 0.63 to 0.93; P = 0.02 for interaction), cerebrovascular disease (HR 0.81; 95% CI 0.61 to 0.83), and congestive heart failure (HR 0.71; 95% CI 0.61 to 0.73; P = 0.006 for interaction).

Figure 4: HR for the primary outcome stratified by the characteristic of the patient. *An event that occurred >90 days before the date of study enrollment. CAD, coronary artery disease; CHF, congestive heart failure; CVA, cerebrovascular accident (stroke); PVD, peripheral vascular disease; EPO, epogen.

Discussion

Using clinical data abstracted from electronic medical records, we modeled a previously published RCT (4D Study) that investigated the cardiovascular benefit of atorvastatin in patients with ESRD and type 2 diabetes. Our observational simulation study found that statin use (versus nonuse) was significantly associated with an 18% decrease in all-cause mortality and a 10% decrease in death from cardiac causes, fatal stroke, nonfatal MI, and nonfatal stroke. The effect sizes were statistically comparable to HRs reported by the 4D Study, which illustrate the potential of well-done observational modeling as an adjunctive tool for clinical investigation.

The practice of evidence-based medicine begins with the appraisal of research design, in which the reliability of results can initially be inferred by the type of study. Only randomized, prospective studies can provide reliable estimates of treatment effect, and for that reason, RCTs are “hypothesis validating” and remain the gold standard in clinical research (³⁰). Although statistical methods can theoretically adjust for identifiable differences between groups, it is impossible to be certain that all pertinent characteristics are measured and can be adequately modeled with linear equations (³¹). For this reason, observational trials are “hypothesis generating” and adjunct to RCT results when available. The reliability of observational study findings and how they should be integrated into clinical/research practice remain an area of debate in the nephrology community. Indeed, the strengths and weaknesses of observational studies and RCTs may be seen as complementary (⁸) in the investigation process, in which observational trials can play an important role in predicting the effectiveness of intervention, for sample size calculation (i.e., estimate the number of subjects for recruitment), and in post-RCT monitoring for rare adverse events.

This simulation of the 4D Study explored the reliability of an observational analysis that incorporates newer epidemiologic instruments to emulate the characteristics of an RCT. In RCTs, study treatment is initiated after patient enrollment to prevent survivorship bias and ensure that baseline characteristics cannot be influenced by the treatment. This approach was modeled in this observational analysis by enrolling only new statin users who had pretreatment baseline measurements (i.e., cholesterol). Residual and nonlinear confounding was minimized through treated–untreated subject matching (1) on the index date to reduce practice drift effects over time, (2) on vintage to decrease ‘”immortal time” bias (³²) (i.e., waiting time from the start of ESRD to study enrollment in both groups were similar), and (3) within the same clinic to balance for unmeasured facility factors. Finally, inverse probability of treatment weighting (i.e., propensity score) reduced selection bias, and the inclusion of a large number of covariates in the survival model was used to account for group differences. As in a prospective trial, the simulation study's inclusion, exclusion, exposure, and outcomes were predeclared exactly as the criteria outlined in the 4D Study. Stratification and dosage-dependent sensitivity analyses were also conducted.

Clinically, this study confirms that patients with chronic kidney disease have higher cardiovascular mortality rates (10 deaths per 100 patient-years) that are similar to patients with coronary disease in the general population (³³); furthermore, when compared with the general population, the prevalence of hypercholesterolemia was relatively low and may be attributed to systemic malnutrition/inflammation that is inherent to ESRD (³⁴). The treatment effect of statins reported in our analysis were consistent with findings of several large HMG-CoA reductase trials of patients without renal disease in that statin use was associated with statistically significant decreases in cholesterol (³⁵) and cardiovascular mortality (^35,36), with greater decreases in patients with coronary artery disease (i.e., secondary prevention); however, the effect sizes were lower when compared with point estimates in the general population. We provide three possible explanations for this discrepancy. First, atherogenesis in ESRD may be mediated by different pathways than outlined for patients without renal failure (³⁷). Given the relatively small reduction in total cholesterol in the statin user group (versus non–statin user group), the association between hypocholesterolemia with mortality in ESRD (^38,39), and depressed LDL values (<130 mg/dl) that are characteristic of patients who are on dialysis (^9,40), it is conceivable that the beneficial effect of statin therapy may be less from cholesterol lowering but more likely exerted through alternative mechanisms. For example, statin use (versus nonuse) was associated with statistically significant increases in albumin in this study, which support the anti-inflammatory hypothesis (⁴¹) for cardiovascular disease progression and its mediation through HMG-CoA reductase inhibitors (^42,43). Second, lower HRs may be secondary to greater statin misclassification associated with medical record–based analysis over information gathered under research protocol. Third, the effectiveness of statins may vary by baseline lipid level (⁴⁴), which was lower in our cohort when compared with the level reported in the 4D Study (⁹) and prospective trials in the general population (^{35,36,45–47}).

We also substantiate an association between statin use and strokes, which was reported as “chance” in the 4D Study and as a “marginal increase” in A study to evaluate the Use of Rosuvastatin in subjects On Regular haemodialysis: an Assessment of survival and cardiovascular events (AURORA) (⁴⁰). This finding is in contrast with general population trials (⁴⁸) and suggests that the pathogenesis for strokes in dialysis-dependent patients may include novel pathways. Further studies are recommended to investigate this trend.

Comparison between the 4D Study and this study also highlight some of the limitations of chart review (versus blinded committee adjudication) and residual selection inherent to observational analyses. Statin users were found to have a nonsignificant 14% increase in nonfatal angina or revascularization (Table 3). The finding highlights residual confounding that could not be adequately balanced with covariate and propensity scoring methods given that patients with preexisting atherosclerosis are more likely to be placed on a statin. We also point out evidence of misclassification associated with retrospective chart review demonstrated by the significant association between statin use and death from an unknown cause (approximately 15% of deaths). Some of these patients may have in fact died from stroke or cardiovascular disease, which may have biased the effect estimate for all-cause mortality. Many patients may not have been fasting during laboratory testing, which may have artificially elevated their cholesterol and glucose levels. We were also unable to assess the accuracy of the data against physician notes, mostly because the cohort was historical and the patient charts were not accessible. Finally, although point estimates of association between the 4D Study and its simulated trial were numerically comparable, we caution against a direct comparison given the wide confidence limits of the 4D Study HRs and some baseline differences between the two cohorts. The study population herein was younger, had a longer vintage, was more likely to be female, had a lower cholesterol level, had less cardiovascular disease, had limited LDL monitoring, and had larger body size than the patients who were enrolled in the 4D Study.

The need for better evidence of comparative clinical effectiveness in ESRD is great, yet high costs limit our ability to investigate all research questions with an RCT (⁴⁹). When an RCT is not immediately realizable, a simulated trial may provide reasonable effect estimates to test a research hypothesis, determine study feasibility, and help leverage funding for future randomized trials. The use of clinical surveillance systems for observational research can also provide certain advantages: It increases external validity because there is less active selection of patients during the study enrollment; when the surveillance population is large, it can facilitate the enrollment of many patients over a relatively short time interval, which reduces bias from the period effect where improvement in practice patterns over time can affect the study outcome; and it reduces costs by providing a means to enroll a large number patients over a wide geographic area.

Conclusions

Clinical trial simulation may represent a complementary instrument for clinical investigators that can provide reasonably accurate effect estimates of treatment. For example, statins in patients with ESRD seemed to be associated with some cardiovascular benefit in a large-scale simulation of the 4D Study; however, the treatment effect is very attenuated and a cholesterol-mediated mechanism seems to play a minor role in cardiovascular disease progression in dialysis patients.

Disclosures

None.

We express appreciation to the staff in more than 1500 Fresenius dialysis clinics who continually make great efforts to ensure the accurate charting of clinical data in the computer system.