Using Quality Improvement Databases to Advance Medical Knowledge: Opportunities and Challenges
Neuman, Mark D. M.D.*; Fleisher, Lee A. M.D.†
Good outcome measures are vital feedback indicating what works and what does not. Every thriving sector of the economy harnesses this kind of information to spur learning. Health care is the outlier.1
Business strategy experts Michael E. Porter and Elizabeth Olmsted Teisberg have recently proposed a model for change in American health care that emphasizes the power of improvements in quality to create a more efficient and less costly system of care. In their view, the systematic assessment of clinical results is fundamental to any such change.1,2
In this issue of Anesthesiology, Kheterpal et al.
draw novel insights into postoperative acute kidney injury (AKI) in general surgical patients from data collected by the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP), a rigorous, validated outcomes monitoring project.3
The work of Kheterpal et al.
casts new light on the risks and outcomes associated with AKI, demonstrating the great potential and the fundamental challenges of using quality-improvement databases to advance medical knowledge.
Porter and Teisberg point to ACS-NSQIP as the type of clinical results database that “should become the norm for the treatment of every medical condition.”2
Originating in an effort to improve the quality of surgical care in U.S. Veterans Administration medical centers,4
ACS-NSQIP has since been adapted for use in a range of nongovernmental institutions.5
Over 200 U.S. medical centers, including community facilities and tertiary care hospitals, now participate voluntarily in the program's structured, prospective monitoring of sampled surgical cases. Trained nurse data collectors obtain detailed information on patient and procedure variables extending from the preoperative period to 30 days postoperatively.5
Risk-adjusted outcome data provided by ACS-NSQIP has been used for quality improvement by participating hospitals and health systems6
; research using aggregated ACS-NSQIP data has provided risk models for the development of specific postoperative complications7,8
and estimates of operative morbidity and mortality in U.S. Veterans' Administration hospitals and university medical centers.9,10
Kheterpal et al.
examine ACS-NSQIP data from more than 150,000 surgical cases performed at 121 sites in 1 yr, describing the risk factors and 30-day mortality associated with AKI after general surgical procedures. Excluding cardiac and vascular operations, where the risk of postoperative renal failure is well established,11
they observe a 1% incidence of AKI after general surgery. Through logistic regression modeling, Kheterpal et al.
define a set of independent predictors of postoperative AKI and derive a prognostic risk score. As they describe, presence of six or more risk factors was associated with an 8.9–9.5% incidence of postoperative renal failure, compared to a 0.2% incidence associated with zero to two risk factors. Development of postoperative AKI was associated with a 30-day mortality of 42%, compared to a control mortality of 8.6% (P
< 0.001), highlighting AKI as a grave prognostic indicator.3
Does the ability to better predict risk help us to improve the quality of care? The answer is complex. Some risk factors for AKI, like gender and age, are not modifiable, and it is unknown if optimization of other risk factors, such as diabetes or hypertension, can reduce the risk of renal injury. Aside from the avoidance of known neprhotoxins, practitioners still lack proven strategies for renal protection in general surgical patients at high risk for AKI.11,12
Nonetheless, the ability to predict risk is an essential step in the assessment and improvement of quality. Improved prediction models will allow patients, practitioners, administrators, insurers, and regulators to make more meaningful comparisons of outcomes between hospitals through better risk-adjustment. Improved tools for risk-stratification will allow future studies of renal-protection interventions to target high-risk patients. Kheterpal's study provides insight into possible avenues for further investigation by highlighting modifiable risk factors for AKI such as hypertension, diabetes, and congestive heart failure.
Kheterpal's analysis relies on the rigorous follow-up of ACS-NSQIP, which provides 30-day outcome data for all patients in the study cohort. As perioperative outcomes research comes to focus on complex, multifactorial outcomes, such as renal failure and postoperative cognitive dysfunction,13
researchers and clinicians alike will require a detailed understanding of clinical results beyond hospital discharge. Reliable methods to monitor outcomes at 30 days and beyond are thus central to any meaningful effort at perioperative quality improvement. As Kheterpal's work demonstrates, the consistent 30-day follow-up of ACS-NSQIP underpins its value as a quality improvement tool and as the basis for meaningful research on complications whose consequences extend beyond the window of hospitalization.
Despite these strengths, the current study is inherently limited by challenges common to any analysis of such a database as the ACS-NSQIP. As with any examination of observational data, associations between risk factors and outcomes in the ACS-NSQIP dataset are subject to the influence of unmeasured confounding variables. The ACS-NSQIP dataset lacks several categories of potentially important patient data, such as intraoperative hemodynamics, fluid management, urine output, and nephrotoxin exposure. Although the associations that can be drawn from ACS-NSQIP data may greatly inform our understanding of a disease process, the potentially confounding effects of unmeasured variables limit our ability to establish causality between risk factors and clinical outcomes.
A further challenge presented by the ACS-NSQIP lies in its design as a quality-improvement tool rather than a protocol-driven research instrument, as the data it collects vary according to patient factors and individual physician practices. Importantly for the present study, a preoperative serum creatinine value was not available for 9.4% of patients. As the definition of AKI here relies, in part, on a rise in creatinine from a preoperative value, the large fraction of patients without baseline measurements represents an important source of potential bias. Through a rigorous analysis, the authors convincingly demonstrate a minimal impact of these missing values on their study results. Still, the challenge of assessing and quantifying the impact of missing information represents a major hurdle in adapting quality-improvement data to research seeking valid answers to study questions.
These challenges are common limitations of research using quality-monitoring databases. Beyond rigorous methods of study design and statistical analysis, the solution to such dilemmas ultimately lies in developing the quantity and quality of data made available through efforts such as ACS-NSQIP. With the development and spread of electronic medical records and anesthesia information management systems, the future of operative outcomes research may lie in the ability to link the detailed measurement of intraoperative variables with rigorous, prospectively collected postoperative information.
In their analysis of ACS-NSQIP data, Kheterpal et al.
thus adapt a powerful quality-measurement tool to advance our understanding of AKI after general surgery. The power of ACS-NSQIP as a research tool and the value of the analysis by Kheterpal et al.
embody important aspects of the promise that Porter and Teisberg envision in the spread of systematic results monitoring in healthcare.1
For anesthesiologists, the strengths of ACS-NSQIP and the work of Kheterpal et al.
should come as a call to support the development of systems to reliably measure patient outcomes beyond the immediate postoperative period. Further, by supporting implementation of anesthesia information management systems and working to link intraoperative data with postoperative outcome measurement, anesthesiologists can contribute to a more complete understanding of the interplay of physiology, intraoperative management, and patient outcomes. Through such efforts, anesthesiology as a specialty has the opportunity to move medicine away from its status as an outlier among developed industries and towards a model of practice in which the rigorous measurement of clinical results fosters learning and contributes to safer patient care.
Mark D. Neuman, M.D.,*
Lee A. Fleisher, M.D.†
*University of Pennsylvania Robert Wood Johnson Foundation Clinical Scholars Program, Philadelphia, Pennsylvania. †Department of Anesthesiology and Critical Care, University of Pennsylvania, Philadelphia, Pennsylvania. firstname.lastname@example.org
1. Porter ME, Teisberg EO: How physicians can change the future of health care. JAMA 2007; 297:1103–11
2. Porter ME, Teisberg EO: Redefining Health Care: Creating Value-Based Competition on Results. Boston, Massachusetts, Harvard Business School Press, 2006.
3. Kheterpal S, Tremper KK, Heung M, Rosenberg AL, Englesbe M, Shanks AM, Campbell DA Jr: Development and validation of an acute kidney injury risk index for patients undergoing general surgery: Results from a national dataset. Anesthesiology 2009; 110:505–15
4. Khuri SF, Daley J, Henderson W, Barbour G, Lowry P, Irvin G, Gibbs J, Grover F, Hammermeister K, Stremple JF, et al: The National Veterans Administration Surgical Risk Study: Risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg 1995; 180:519–31
5. Khuri SF, Henderson WG, Daley J, Jonasson O, Jones RS, Campbell DA Jr, Fink AS, Mentzer RM Jr, Neumayer L, Hammermeister K, Mosca C, Healey N: Successful implementation of the Department of Veterans Affairs' National Surgical Quality Improvement Program in the private sector: The Patient Safety in Surgery study. Ann Surg 2008; 248:329–36
6. Khuri SF: The NSQIP: a new frontier in surgery. Surgery 2005; 138:837–43
7. Rogers SO Jr, Kilaru RK, Hosokawa P, Henderson WG, Zinner MJ, Khuri SF: Multivariable predictors of postoperative venous thromboembolic events after general and vascular surgery: Results from the patient safety in surgery study. J Am Coll Surg 2007; 204:1211–21
8. Arozullah AM, Khuri SF, Henderson WG, Daley J: Development and validation of a multifactorial risk index for predicting postoperative pneumonia after major noncardiac surgery. Ann Intern Med 2001; 135:847–57
9. Fink AS, Hutter MM, Campbell DC Jr, Henderson WG, Mosca C, Khuri SF: Comparison of risk-adjusted 30-day postoperative mortality and morbidity in Department of Veterans Affairs hospitals and selected university medical centers: General surgical operations in women. J Am Coll Surg 2007; 204:1127–36
10. Henderson WG, Khuri SF, Mosca C, Fink AS, Hutter MM, Neumayer LA: Comparison of risk-adjusted 30-day postoperative mortality and morbidity in Department of Veterans Affairs hospitals and selected university medical centers: General surgical operations in men. J Am Coll Surg 2007; 204:1103–14
11. Sear JW: Kidney dysfunction in the postoperative period. Br J Anaesth 2005; 95:20–32
12. Jones DR, Lee HT: Perioperative renal protection. Best Pract Res Clin Anaesthesiol 2008; 22:193–208
13. Moller JT, Cluitmans P, Rasmussen LS, Houx P, Rasmussen H, Canet J, Rabbitt P, Jolles J, Larsen K, Hanning CD, Langeron O, Johnson T, Lauven PM, Kristensen PA, Biedler A, van Beem H, Fraidakis O, Silverstein JH, Beneken JE, Gravenstein JS: Long-term postoperative cognitive dysfunction in the elderly ISPOCD1 study. ISPOCD investigators. International Study of Post-Operative Cognitive Dysfunction. Lancet 1998; 351:857–61
© 2009 American Society of Anesthesiologists, Inc.
Publication of an advertisement in Anesthesiology Online does not constitute endorsement by the American Society of Anesthesiologists, Inc. or Lippincott Williams & Wilkins, Inc. of the product or service being advertised.