A Race against Time: Planning Postoperative Critical Care
Mazo, Valentín M.D.*; Sabaté, Sergi M.D., Ph.D.†; Canet, Jaume M.D., Ph.D.*
Photo: Thinkstock....Image Tools
“Understanding the weight of these events will enable us to confirm or correct predictions made during preoperative assessment, so that we can make faster decisions about complicated surgical patients.”
CRITICAL care planning for surgical patients is not an easy task because it involves anticipating the risk of adverse effects in most of the vulnerable patients we have treated. The difficulty arises from the unpredictability of complications that emerge both during surgery and afterwards in patients admitted to critical care units for periods that may extend over many days and weeks. Wherever resources are already stretched to the limit, decision-making becomes more burdensome because of the high cost of critical care.
In this issue of ANESTHESIOLOGY, Wanderer et al
propose an intraoperative prediction model for unplanned admission to the critical care unit. The authors examined more than 70,000 anesthetic records with more than 4,500 events (unplanned admissions), giving them sufficient leeway to test a large number of hypothetical predictors. Through bootstrapping, they provided assurance against overfitting of the model and strengthened the finding of internal validity. The performance of the model was compared with that of the Surgical Apgar Score2
using an interesting series of time-dependent receiver-operating characteristic curves with an acceptable discrimination value at 1 h before case end. Sensitivity, specificity, and predictive values of the model were calculated for a threshold of 5% of predicted probability of unplanned admission. The optimal cut points based on the areas under the curves can be disputed, as the authors do not report the criteria for their choice. Alternative cutoffs might well be considered in the interest of avoiding excessive dichotomization and establishing an intermediate gray zone, as suggested by Ray et al
Furthermore, because predictive values can be severely affected by prevalence, likelihood ratios would have shed further light.3
In this study, a positive and negative likelihood ratio of approximately 4 and 0.17, respectively, have a moderate degree of predictive usefulness.
The nine predictors found to be associated with hemodynamic events are consistent with what others have emphasized, that hemodynamic instability appears to play a key role in postoperative outcome.4–9
Arterial oxygen saturation as measured by pulse oximetry (specifically, a decrease in the ratio of arterial oxygen saturation to the fraction of inspired oxygen) was among the predictors, consistent with its recently identified implication in respiratory complications.10
Three predictors are factors well known before surgery (American Society of Anesthesiologists class, age, and high-risk surgery); but although these factors can be considered in preoperative risk prediction and critical care planning, they may also bear a relation to intraoperative complications. Emergency operation proved to be another predictor of unplanned admission, unsurprisingly, because it is recognized that emergency status involves higher risk. But the fact that Wanderer et al
classified admissions as unplanned for all patients who underwent an emergency operation deserves a comment: it is anomalous to define a candidate predictor in terms of the dependent variable, the outcome of interest. When a factor is defined in this way, its involvement in the model may be overestimated. Although that categorical decision may be logical from an administrative standpoint within a hospital, from a medical point of view the destination of many emergency patients could be planned before surgery, and if these cases had been defined differently in this study, the model may well have been different.
The longer we watch a patient and weigh the information at hand, the more accurately we can predict. But a prediction that comes late, too close to the event we anticipate, will be less useful than one that comes earlier. Once we are certain, and the need for an immediate response is at hand, prediction is no longer an issue. Under normal conditions, the best time for risk prediction is during the preoperative visit. There, the patient can be duly informed and the anesthesiologist can prescribe preventive measures and foresee postoperative care requirements. Accumulated experience tells us that some intraoperative factors are known or can be anticipated in this visit. Examples are type of intervention, surgical incision, aggressiveness, or expected duration of surgery. Clearly, unusual intraoperative events (complications) cannot be taken into account when planning: such events are weighed in the operating room with a view to change the care plan as needed. When unplanned admission to a critical care unit becomes necessary, events have unfolded in a way that unfortunately leaves little room for choice. The outcome specified by Wanderer et al
—unplanned admission—is a truly complex real-life event because such admission involves a complication followed by a clinical decision. It seems logical to assume that decisions may be tightly constrained by critical care admission criteria, which may be more or less restrictive in different settings. The criteria operating in this study are difficult to determine, however, given that data were gathered retrospectively and it may be that future external validation of the model in different hospital systems could provide different results.
Whenever a multivariable analysis of risk factors is attempted, we begin to debate whether we have an explanatory or a predictive model. An ideal model would be one that gives us as much explanatory information as possible but that also predicts reasonably well. Because it is too bold to infer causal relationships from associations in observational studies, models are usually assumed to be predictive and their explanatory value depends on the biological plausibility of the predictors. Paradoxically, although we may manage to foresee events based on modeled associations, prediction in the narrow range of 1 or 2 h will probably be of limited practical use for hospitals with poor resources, where more time for maneuvering is a key to success.
After a predictive model has been built by multivariable analysis and internally validated by assessment of its ability to identify the individuals who will develop the outcome as well as the congruence between the predicted and observed frequencies, we must next ask about the future use of this model. Predictive models make sense only if they are useful in upcoming situations. If a model has been ideally based on a heterogeneous sample aspiring to represent a whole population, we can hope that it will be externally validated more easily. In contrast, a model based on a highly specific sample is less likely to be transportable. In any case, the proof of the pudding is in the eating as the old saying goes. Therefore, we should always plan as robust as possible an external validation of any model to verify the limits of its usefulness and establish a hierarchy of models based on their performance.11
What is needed is validation in different geographical settings, by independent research groups and even at different historical moments. Furthermore, even in a well-constructed predictive model, recalibration may be necessary to obtain an acceptable performance if the model is applied in highly selective contexts or ones that are different in any way from those of the development sample (e.g
., pediatric medicine, tertiary hospitals, particular ethnic groups and so on).12
Nowadays, despite advances in computational and statistical methods in the health sciences, we are cautious about searching for a predictive score that will be useful everywhere and forever. Continuous adjustments are necessary to optimize these tools.
We conclude by emphasizing that the model developed by Wanderer et al
shows the importance of intraoperative events as predictors of patients’ admission to critical care units. Understanding the weight of these events will enable us to confirm or correct predictions made during preoperative assessment, so that we can make faster decisions about complicated surgical patients. However, we encourage other researchers and clinicians to study this tool further to substantiate its transportability. The more evidence we have of the utility of any model in different settings, the more our theories will begin to meet the needs of practice.
1. Wanderer JP, Anderson-Dam J, Levine W, Bittner EA. Development and validation of an intraoperative predictive model for unplanned postoperative intensive care. ANESTHESIOLOGY. 2013;;119:516–24
2. Gawande AA, Kwaan MR, Regenbogen SE, Lipsitz SA, Zinner MJ. An Apgar score for surgery. J Am Coll Surg. 2007;204:201–8
3. Ray P, Le Manach Y, Riou B, Houle TT. Statistical evaluation of a biomarker. ANESTHESIOLOGY. 2010;112:1023–40
4. Charlson ME, MacKenzie CR, Gold JP, Ales KL, Topkins M, Fairclough GP Jr, Shires GT. The preoperative and intraoperative hemodynamic predictors of postoperative myocardial infarction or ischemia in patients undergoing noncardiac surgery. Ann Surg. 1989;210:637–48
5. Charlson ME, MacKenzie CR, Gold JP, Ales KL, Topkins M, Shires GT. Intraoperative blood pressure. What patterns identify patients at risk for postoperative complications? Ann Surg. 1990;212:567–80
6. Barone JE, Bull MB, Cussatti EH, Miller KD, Tucker JB. Perioperative myocardial infarction in low-risk patients undergoing noncardiac surgery is associated with intraoperative hypotension. J Intensive Care Med. 2002;17:250–5
7. Röhrig R, Junger A, Hartmann B, Klasen J, Quinzio L, Jost A, Benson M, Hempelmann G. The incidence and prediction of automatically detected intraoperative cardiovascular events in noncardiac surgery. Anesth Analg. 2004;98:569–77
8. Monk TG, Saini V, Weldon BC, Sigl JC. Anesthetic management and one-year mortality after noncardiac surgery. Anesth Analg. 2005;100:4–10
9. Reich DL, Bennett-Guerrero E, Bodian CA, Hossain S, Winfree W, Krol M. Intraoperative tachycardia and hypertension are independently associated with adverse outcome in noncardiac surgery of long duration. Anesth Analg. 2002;95:273–7
10. Canet J, Gallart L, Gomar C, Paluzie G, Vallès J, Castillo J, Sabaté S, Mazo V, Briones Z, Sanchis JARISCAT Group. . Prediction of postoperative pulmonary complications in a population-based surgical cohort. ANESTHESIOLOGY. 2010;113:1338–50
11. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130:515–24
12. Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol. 2008;61:76–6
13. Moons KG, Kengne AP, Grobbee DE, Royston P, Vergouwe Y, Altman DG, Woodward M. Risk prediction models: II. External validation, model updating, and impact assessment. Heart. 2012;98:691–8
© 2013 American Society of Anesthesiologists, Inc.
Publication of an advertisement in Anesthesiology Online does not constitute endorsement by the American Society of Anesthesiologists, Inc. or Lippincott Williams & Wilkins, Inc. of the product or service being advertised.