Differences in mortality rates associated with coronary artery bypass graft (CABG) surgery have been well documented, but solely observing these differences does not necessarily suggest opportunities for improvement (1–3). In contrast, identifying the etiology of these observed differences permits development of specific strategies and interventions to improve mortality, through reduction of variation in mortality rates.
The Northern New England Cardiovascular Disease Study Group (NNECDSG) has recently gained knowledge about the etiology of differences in mortality rates after CABG surgery (4). First, heart failure was the most common etiology of death among CABG patients. Second, when groups of surgeons with larger (5.8%) and smaller (3.2%) mortality rates were compared, 80% of the variation in risk adjusted mortality rates was explained by differences in the incidence of heart failure as the seminal event leading to death.
Based on this knowledge, the NNECDSG has focused most of its continuing quality improvement efforts toward the recognition of heart failure. Our hypothesis is that improved vigilance for heart failure will lead to reduced regional mortality because this strategy is aimed specifically at the etiology of variation in mortality rates. The purpose of this study was to develop a clinical risk assessment tool that anesthesiologists, surgeons, and other clinicians can use to rapidly and easily assess risk of fatal heart failure while caring for individual CABG patients.
Methods
NNECDSG is a regional voluntary consortium with the mission to develop and exchange information concerning treatment of cardiovascular disease. The NNECDSG consists of clinicians, hospital administrators, and health care research personnel who seek to continuously improve the quality, safety, effectiveness, and cost of medical interventions in cardiovascular disease. The Group maintains continuing prospective registries of all patients receiving percutaneous transluminal coronary angioplasty, CABG and heart valve repair or replacement surgery since 1987 at the five original centers. Currently eight medical centers prospectively collect data in NNECDSG registries. These centers are: Eastern Maine Medical Center, Bangor, ME; Maine Medical Center, Portland, ME; Catholic Medical Center, Manchester, NH; Dartmouth-Hitchcock Medical Center, Lebanon, NH; and Fletcher Allen Health Care, Burlington, VT; Beth Israel Deaconess Medical Center, Boston MA; Concord Hospital, Concord, NH; and Portsmouth Regional Hospital, Portsmouth, NH.
Data for the present study were also used for the NNECDSG Regional Study of Modes of Death Associated with Coronary Artery Bypass Grafting (4). The following is a summary of the methodology used in this prior study to assign mode of death (4). In-hospital deaths during the study period were reported by the cardiac surgeons and were validated using administrative data provided by the participating hospitals. Mode of death was assigned as heart failure, hemorrhage, dysrhythmia, respiratory failure, neurologic causes, or other cause. The intent of the assignment process was to identify the seminal clinical event leading to each patient’s death. Coding of death was accomplished using rules predefined by the clinical endpoints committee, which consisted of two cardiac surgeons, a cardiac surgery fellow, and an epidemiologist. The committee was blinded to all preoperative patient data. Heart failure was identified as the mode of death if management of hypotension and/or low cardiac index (with return-to-bypass, inotropic support, and/or intraaortic balloon pump) was the seminal event leading to death. This seminal event could differ from the terminal condition leading directly to death. For example, one patient experienced a severe cerebrovascular accident on postoperative Day 2. The patient died 2 wk later from aspiration pneumonia. The mode of death for this patient was classified as neurologic, not respiratory. One-hundred patients who died were randomly selected for reevaluation. The committee was unaware of this reevaluation process. The kappa statistic used to assess the concordance between repeat classifications was 0.88 (P < 0.0001), indicating excellent agreement (5,6).
Five of the medical centers in the NNECDSG participated in the current study and the Regional Study of Modes of Death Associated with Coronary Artery Bypass Grafting (4). These centers were Eastern Maine Medical Center, Maine Medical Center, Catholic Medical Center, Dartmouth-Hitchcock Medical Center, and Fletcher Allen Health Care. Data were prospectively collected on 8641 consecutive patients having isolated CABG procedures between July 1, 1987 and April 1, 1991. Patients who had isolated valve or combined valve/CABG surgery were excluded from this study.
Preoperative characteristics included age, sex, diabetes, chronic obstructive pulmonary disease, peripheral vascular disease, dialysis dependent renal failure and prior CABG surgery. Cardiac catheterization results included ejection fraction, left ventricular end diastolic pressure, number of diseased coronary arteries, and stenosis of the left main coronary. Priority at surgery was classified as emergency, urgent, or elective. Emergency surgery required medical factors relating to the patient’s cardiac disease dictate that surgery should be performed within hours to prevent morbidity or death. Urgent surgical status implied that medical factors require the patient to stay in the hospital to have the operation before discharge. Variables used in the present study were similar to those used by other investigators (7).
The outcome we examined was death as a result of heart failure. Preoperative patient characteristics showing a significant association with fatal heart failure in univariate analyses were entered into a multivariate logistic regression model and nonsignificant variables were eliminated (Stata Statistical Software, release 5.0; Stata Corporation, College Station, TX). The area under the receiver operating characteristic (ROC) was used as a measure of the discrimination of the logistic regression model (8). Goodness-of-fit for the regression model was examined using the Hosmer-Lemeshow test (9). A low χ2 value and a high nonsignificant P value for the Hosmer-Lemeshow test indicated that the fit of the model and observed values was acceptable.
Validation of the prediction model was completed internally using bootstrap re-sampling techniques (200 samples of 70% of the data) to estimate the bias corrected 95% confidence intervals of the ROC area (10). Bootstrapping was selected instead of cross-validation or data splitting because bootstrapping provides nearly unbiased estimates of predictive accuracy that are of relatively small variance (11).
A clinical risk assessment tool was developed from the multivariate logistic regression model and was scored by rounding the adjusted odds ratio for each variable. These weights were then summed. The relationship between this clinical risk score and the probability calculated from the logistic regression model was read from a graph. This risk assessment tool therefore approximates the risk that would have been calculated from the logistic regression equation. The area under the ROC curve and the Hosmer-Lemeshow test were used to evaluate the discrimination and goodness of fit of this risk assessment tool.
Results
There were 387 deaths among 8641 patients (4.48%), and of these, 64.8% of the deaths (n = 249) were attributed to heart failure in this prospective multicenter observational study. The logistic regression model was statistically significant (χ2[11 df] = 197.2, P < 0.001). Advanced age, female sex, diabetes, peripheral vascular disease, renal failure, depressed ejection fraction, number of diseased coronary vessels, prior CABG surgery, and urgent or emergency surgery were associated with increased risk of fatal heart failure (Table 1). The ROC area was 0.76 (95% confidence interval: 0.72, 0.78). The Hosmer-Lemeshow test was nonsignificant (χ2LH [3 df] = 5.69, P = 0.128) and the correlation between predicted and observed events by quintile of predicted risk was 0.96 (P < 0.001).
;)
Table 1: Predictors of Fatal Heart Failure in Coronary Artery Bypass Graft (CABG) Patients
The clinical risk assessment tool is presented in the legend of Table 1. The sum of each weight determines the score for an individual patient’s risk of fatal heart failure. For each discrete score, the predicted risk was calculated using the multivariate regression model (Fig. 1). This clinical score had discrimination ability and confidence intervals similar to the logistic regression based prediction model (ROCclinical score: 0.75, 95% confidence interval: 0.71, 0.78). The Hosmer-Lemeshow test for the clinical risk assessment tool was also nonsignificant. The correlation between the predicted risk obtained from the logistic regression equation and that obtained from the risk assessment tool was 0.99 (P < 0.001).
Figure 1: Predicted risk of fatal heart failure by clinical risk score. Using the weights from Table 1 allows the rapid approximation of the probability of heart failure death. The patient is a 75-yr-old woman with diabetes and depressed left ventricular function. Her total clinical risk score is 7 (1.5 for age 70–79 + 1.5 for female sex + 1.5 for diabetes + 2.5 for left ventricular ejection fraction <40%.
A pocket-sized card for the clinical risk score is presented in Figure 2. The clinical risk assessment tool has a possible range from zero (i.e., a 30-yr-old male for elective CABG) to 23 (i.e., an 80-yr-old female for emergent redo CABG who has all patient and disease characteristics that contribute to the score). In these data the observed range of scores was from 0 to 18.5. Patients with clinical risk scores between 0 and 3 are in the bottom 45.5% of risk, whereas those with clinical risk scores of more than or equal to 7 are in the top 10% of risk of death from heart failure. In these data only 1% of patients have clinical risk scores more than 10.
Figure 2: Pocket-sized card for clinical risk assessment tool.
Discussion
Prospectively collected preoperative data were used to develop a multivariate prediction rule for fatal heart failure. The logistic regression equation provides a method of calculating risk of fatal heart failure and consequently a method to adjust for case mix differences in comparing different approaches to care. A clinical risk assessment tool was derived from this multivariate equation. The discriminatory characteristics of the multivariate prediction equation and the risk assessment tool were satisfactory and their parameterization was excellent.
Other cardiac surgical scoring systems have been reported (1–3). Although the factors that contribute to the present model are similar to those reported in previous scoring systems, the current scoring system is unique. First, this scoring system identifies not the risk of death, but instead the risk of death resulting from heart failure after CABG surgery. This distinction is crucial, because most variation in the mortality rates across our region is explained by the incidence of fatal heart failure (4). Second, there are differences in the odds ratios associated with preoperative variables as compared with other models (12). These differences are relevant when the clinician is evaluating the risk of postoperative heart failure while caring for an individual CABG patient.
Three limitations deserve mention. First, this is a regional study and these results should be confirmed in other geographic areas. Although validation of the logistic model and clinical risk assessment tool in another group of patients would be valuable, no other cardiac surgical database exists with information on mode of death and adequate sample size. The bootstrapping methodology used for validation in this study has advantages compared to other validation options such as cross-validation or data splitting. Bootstrapping is more efficient because the entire data set is used for model development (10,11). The NNECDSG continues to prospectively collect mode of death data and in the future will report the results of a regional approach to improving the mortality associated with CABG that uses this clinical risk score to focus attention on low output failure. One prospective study by Rao et al. (13) examined the association of patient characteristics to an outcome similar to heart failure as a mode of death. They defined low cardiac output as the requirement for postoperative intraaortic balloon pump or prolonged inotropic support to maintain systolic blood pressure more than 90 mm Hg and cardiac index more than 2.2 L/min per square meter. Similar predictors to those described in the present study were observed by Rao et al. (13).
Second, even in a large database there will be relatively few patients at the highest risk scores. We recommend that a patient with clinical risk score more than 10 be considered to have a greater than 15% risk of heart failure death. In our practice, we consider any patient with a score more than or equal to seven to be at high risk of fatal heart failure. Use of a score of seven as a cutoff identifies patients in the top 10% of risk of death from heart failure.
Third, the clinical risk assessment tool is less precise than the predicted risk calculated directly from the logistic regression equation. This is a consequence of rounding off the adjusted odds ratios to derive the weights for the clinical risk assessment tool. Based on the ROC area and the Hosmer-Lemeshow test results, the clinical risk assessment tool is an acceptable alternative to the logistic regression. The advantage of the clinical risk assessment tool is that it is less intimidating to the majority of clinicians compared to bedside application of a logistic regression model.
To facilitate clinicians, this clinical risk assessment tool is printed on pocket-sized cards and is currently used at the bedside during clinical decision making by physicians at the 8 hospitals (Fig. 2). The philosophy of the NNECDSG is that improvement activities are initiated by clinicians rather than by nonclinicians. This ensures control of the improvement effort is held by those most likely to make meaningful changes in actual care. Each participating center in the region has an established multidisciplinary cardiac surgical team that is responsible for development and implementation of specific clinical interventions that will be most effective, recognizing the unique aspects of their process of care. As a result, different strategies are being used across the region with the common goal to reduce both the incidence and case fatality rate of heart failure associated with CABG surgery. The following is a summary of general concepts based on a review of published literature regarding the management of heart failure that have been presented at past NNECDSG regional meetings.
Among high-risk patients, an aggressive preemptive strategy aimed at prevention, diagnosis, and treatment of heart failure may improve outcome. The clinical risk assessment tool is presently used to identify need for invasive hemodynamic monitoring and transesophageal echocardiography (14–16). The risks of intraaortic balloon pump counterpulsation (e.g., thrombocytopenia, limb ischemia, aortic dissection) may be outweighed by the benefit of reducing the demand of ischemic myocardium among these patients. Glutamate, glucose, insulin, and potassium infusions have been observed to have a frequent rate of success in treating perioperative heart failure, and one NNECDSG center is currently performing a randomized controlled trial of this technique as a component of their improvement effort (16).
The risk score also identifies patients with small preoperative risk of heart failure. An aggressive process of care aimed at prevention and treatment of heart failure may be unnecessary, or even detrimental to these patients. The distribution of risk of fatal heart failure demonstrates that this small-risk group of patients is a large proportion of the CABG population. Decreased use of aggressive monitoring and therapy in small risk patients may reduce exposure to the risks of heart failure therapies, potentially reducing cost, morbidity, and perhaps mortality of small risk patients. One NNECDSG center is using the clinical risk score to reduce inotrope use after bypass among small risk patients. Another NNECDSG center has reduced the use of pulmonary artery catheters among these patients (18).
In conclusion, we report the development of a clinical risk assessment tool that is printed on pocket-sized cards and is currently used at the bedside during clinical decision making. In contrast to previous cardiac surgical scoring systems that predict total mortality, this clinical risk assessment tool focuses attention on risk of fatal heart failure. This distinction is relevant for quality improvement initiatives, because the majority of the variation in CABG mortality rates is explained by postoperative heart failure.
References
1. Shroyer AL, Plomondon ME, Grover FL, Edwards FH. The coronary artery bypass risk model: the society of thoracic surgeons adult cardiac national database. Ann Thorac Surg 1999;1996:67:1205–8.
2. Hannan EL, Kilburn H, Racz M, et al. Improving the outcomes of coronary artery bypass surgery in New York state. JAMA 1994; 271: 761–6.
3. O’Connor GT, Plume SK, Olmstead EM, et al. Multivariate prediction of in-hospital mortality associated with coronary artery bypass graft surgery. Circulation 1992; 85: 2110–8.
4. O’Connor GT, Birkmeyer JD, Dacey LJ, et al. Results of a regional study of modes of death associated with coronary artery bypass grafting. Ann Thoracic Surg 1998; 66: 1323–8.
5. Cohen J. A coefficient of agreement for nominal scales. educational and psychological measurement. Educational and Psychological Measurement 1960; 20: 37–46.
6. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33: 159–74.
7. Jones RH, Hannan EL, Hammermeister KE, et al. Identification of preoperative variables needed to adjust short-term mortality after coronary artery bypass grafting. J Am Coll Cardiol 1996; 23: 1478–87.
8. Swets JA. Measuring the accuracy of diagnostic systems. Science 1988; 240: 1285–93.
9. Hosmer DW, Lemeshow S. Applied logistic regression. New York, John Wiley & Sons, Inc., 1989.
10. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Statistical Science 1986; 1: 54–77.
11. Harrell FE, Lee KL, Mark DB. Multivariate prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996; 15: 361–87.
12. Eagle KA, Guyton RA, Davidoff R, et al. ACC/AHA guidelines for coronary artery bypass graft surgery: executives summary and recommendations. Circulation 1999; 100: 1464–80.
13. Rao V, Ivanov J, Weisel RD, et al. Predictors of low cardiac output syndrome after coronary artery bypass. J Cardiovasc Surg 1996; 112: 38–51.
14. Pulmonary Artery Catheter Consensus Conference Participants. Pulmonary artery catheter consensus conference: consensus statement. New Horiz 1997; 5: 175–94.
15. Practice guidelines for pulmonary artery catheterization. Anesthesiology 1993; 78: 380–94.
16. Practice guidelines for perioperative transesophageal echo-cardiography: a report by the American Society of Anesthesiologists and the Society of Cardiovascular Anesthesiologists Task Force on Transesophageal Echocardiography. Anesthesiology 1996; 84: 986–1006.
17. Svedjeholm R, Hulijebrant I, Hakanson E, Vanhanen I. Glutamate and high-dose glucose-insulin-potassium (GIK) in the treatment of severe cardiac failure after cardiac operations. Ann Thorac Surg 1995; 59: S23–S30.
18. Stewart RD, Psyhojos T, Lahey SJ, et al. Central venous monitoring use in low risk coronary artery bypass grafting. Ann Thorac Surg 1998; 66: 1306–11.
