Secondary Logo

Journal Logo

Pro and con debate: PONV risk scores

Risk scores for predicting postoperative nausea and vomiting are clinically useful tools and should be used in every patient: Pro – ‘Don't throw the baby out with the bathwater’

Pierre, Sébastien

Author Information
European Journal of Anaesthesiology: March 2011 - Volume 28 - Issue 3 - p 160-163
doi: 10.1097/EJA.0b013e328342fd86

Abstract

Introduction

Post-operative nausea and vomiting (PONV) is a major concern for patients undergoing general anaesthesia1 and remains unacceptably frequent despite the recent publication of North American2 and European3 clinical guidelines. However, many questions regarding PONV have now been answered as a result of the work of teams from, among other countries, Germany, Finland, Switzerland and the United State of America. In summary, three major classes of antiemetics are effective in reducing the incidence of PONV by nearly 30%: corticosteroids (dexamethasone), serotonin (5-hydroxytryptamine type 3) antagonists and neuroleptics (droperidol). Total intravenous anaesthesia (TIVA) using propofol with air also has an antiemetic effect, and the effects of all classes of drug are additive.4,5 This understanding allows us, in cooperation with our surgical colleagues and pharmacists, to choose appropriate interventions to prevent PONV using a risk score-dependent antiemetic strategy.

Unfortunately, spontaneous compliance with guidelines is a disappointing 40%.6,7 To offset this fact, some experts have suggested a shift from the recommended risk-adaptive approach to ultra-liberal administration of antiemetics irrespective of the patient's risk of PONV.8,9 There is no doubt that their purpose is clearly compassionate as many of these experts were involved in the development or validation of risk scores for predicting PONV in adults and in children,10–12 and if barriers to widespread application of clinical recommendations are greater than to the routine use of risk scores, their arguments deserve attention. The first argument made by the proponents of ultra-liberal use of antiemetics is the apparent inherent weakness of clinical risk scores in predicting PONV in an individual patient. Second, there seem to be difficulties related to implementation of risk-score based algorithms. The final point is the safety profile of antiemetics.

Risk assessment is based on the relative impact of true, independent risk factors with clinical importance and it does not matter whether the factors have a causal relationship to the outcome. The most predictive risk factors for PONV in adults are patient-related (female gender, non-smoking status, history of motion sickness or previous PONV), and risk is even greater in association with the use of volatile anaesthetic agents and opioid analgesics.13 This knowledge allows clinicians to stratify patients into groups that are at risk (or at increased risk) and those who are at low risk. The effect of the type of surgical procedure on the incidence of PONV is still debated, at least in terms of clinical significance.14

Unfortunately, the human mind is unable to compute the relative influence of more than two or three risk factors at a time15 and taking into account only one of the patient's risk factors may lead to undertreatment of more than 50% of patients who will suffer from PONV. Consequently, risk scores were developed that combine many risk factors for an outcome into a single predictive measure. In order to be incorporated successfully into clinical practice, predictive models need to be simple. Collecting data on different variables and entering the values into a computer in order to decide what action to take makes sense in clinical research, but is not feasible in clinical practice. Three such ‘simplified’ scores for PONV risk are available in adult or paediatric populations.11,16,17 In the adult setting, a score often referred to as the Apfel score, which is the result of a collaborative study between Finland and Germany, is widely recommended because it can be memorised easily. When none, one, two, three or four factors are present (female gender, history of PONV or motion sickness, non-smoking status, post-operative use of opioids), the risk of PONV is predicted to be approximately 10, 20, 40, 60 or 80%.17

Performance of predictive models

The performance of a predictive model can be quantified in terms of discrimination and calibration. Discrimination applies to measures such as sensitivity, specificity and the area under the receiver operating characteristic (ROC) curve. Regrettably, less than 10% of physicians correctly understand, or are able to use, sensitivity and specificity in clinical practice.18 The understanding of area under the ROC curve (AUC) is a further challenge. The ROC curve plots the sensitivity (true positive rate) against (1 – specificity) (false positive rate) for consecutive cut-offs for the probability of an outcome. The area under the ROC curve is the probability that, for a randomly selected pair of patients, the patient with the event is given a higher risk by the prediction model (Fig. 1).19 This measure depends on the population tested (the more homogeneous the population, the closer the AUC will be to 50%, i.e. equivalent to tossing a coin) and on the relative weight of each risk factor represented by its odds ratio20,21 (http://www.wolfson.qmul.ac.uk/rsc/). This means that, for clinical predictive models, the AUC cannot be more than 70–80% even when additional predictors are considered, unless stronger predictors can be found.22 However, my argument is that this does not matter at all. First, the physician is never faced with a pair of patients and told to find the patient who will suffer from PONV and the patient who will not. Second, the AUC is entirely unaffected by whether or not a prediction is a good one; models with the same discriminating properties may give predictions that are far from the patient's true risk.23

Fig. 1
Fig. 1

Calibration is concerned directly with the estimated probabilities or predictive values. The positive predictive value is defined as the probability of disease given a positive test result. When a risk score is used, the continuous analogue is the probability of disease given the value or range of the score. An assessment of calibration directly compares the observed and predicted probabilities using weighted linear regression analysis when groups of patients are unequal. For example, if we predict a 40% risk of PONV, the observed frequency should be approximately 40 of 100 patients with such a prediction. A graphical assessment of calibration can be made, with predictions on the x-axis and the outcome on the y-axis. Perfect predictions should be on the line of identity. The calibration curve (y = ax + b) can be characterised by an intercept a, which indicates the extent that predictions are systematically too low or too high, and a calibration slope b, which should be 1 (Fig. 2).19 Calibration properties of the three available simplified risk scores are good enough for clinical purposes even when tested in other populations.10,12,19 This last message is crucial. Validation in populations with characteristics that are different to those of the source population demonstrates that the risk score can be generalised and is robust. Otherwise, attempts to draw conclusion about external validity in different settings, for example, after administration of antiemetics or propofol, may lead to inaccurate results.24,25

Fig. 2
Fig. 2

Finally, a very recent study showed that an educational strategy based on encouragement to apply systematically the pre-operative measurement and recording of the simplified Apfel score, together with an emphasis on current guidelines, is successful in decreasing the incidence of PONV in a population of adult surgical patients.26

Implementing predictive models

The second argument against the routine use of risk scores concerns the apparent barriers to implementation of predictive models in clinical practice. However, the studies available about implementation of clinical guidelines for PONV6,7 concern only clinical decision rules and not clinical prediction rules. The difference is subtle, but important. Clinical prediction rules use clinical findings (e.g. history, physical examination and test results) to make a diagnosis or predict an outcome. Clinical decision rules have a supplementary objective, which is to change clinical behaviour and reduce unnecessary cost while maintaining quality of care.27 Standards of evidence for developing and evaluating clinical rules, proposed by the Evidence-based Medicine Working Group, include a four-step approach: derivation of prediction rule, narrow validation, broad validation and narrow impact analysis. The simplified risk score of Apfel et al. and the derived algorithms have satisfied all these requirements.10,17,19,28 A fifth step can be added: broad impact analysis.29 This has been applied in part recently by a German modelling study.9 The results of this study relating to the use of a universal algorithm seem to be unfavourable, but it is important to note some errors in the modelling. Our clinical decision rules28 result in a patient being allocated to one of three groups based on the numbers of risk factors (0 or 1, 2, 3 or 4). The antiemetic strategy that we have proposed comprises administration of no antiemetic to the low-risk patient, one antiemetic to the medium-risk patient and two antiemetics plus TIVA to those in the highest risk group. Judicious use of propofol anaesthesia decreases the risk of PONV by the equivalent of the use of one antiemetic, without increasing the risk of side-effects or the quantity of antiemetics administered.4 This strategy is the only one that satisfies the a priori criteria of efficiency defined in the methods section of the study, regardless of the distribution of risk factors in the population (Fig. 3).19,28,30 Consequently, careful allocation of antiemetics and the use of propofol in a risk-adapted manner have been shown to result in efficacy and efficiency.

Fig. 3
Fig. 3

Would an ultra-liberal treatment approach result in better adherence by physicians? First, administration of, for example, three antiemetics to all patients leads inevitably to a significant reduction in the incidence of PONV regardless of the population studied; this practice is nothing more than another algorithm. Second, barriers to adherence to clinical practice guidelines encompass several components:31 barriers to physician adherence (lack of awareness, lack of familiarity, lack of agreement, lack of self-efficacy, lack of outcome expectancy and inertia of previous practice) and external barriers (guideline-related, patient-related and environment-related barriers). The difficulties related to the use of algorithms is just a small part of the clinical guideline problem, and one that cannot be solved with a ‘magic wand’. At the present time, there is no data that demonstrate that the liberal use of antiemetics is not subject to these same difficulties.

Side-effects

The final argument against the use of risk-score based algorithms is the relatively positive safety profile and the low acquisition cost of antiemetics. Even if we can be very confident while using propofol, dexamethasone and droperidol,32,33 some major concerns arise about the use of serotonin antagonists34 (http://www.hc-sc.gc.ca/dhp-mps/medeff/advisories-avis/prof/_2006/anzemet_nth-aah-eng.php). Furthermore, overuse of any drug without evidence of benefit increases the risks of drug administration errors35 and infection.36

Conclusion

In conclusion, a risk-adapted preventive strategy for PONV is feasible and efficient for nurses and anaesthesiologists and should be offered to most surgical patients, taking into account local constraints and each patient's preferences. Clinical behaviour will change only if we disseminate our enthusiasm and promote and explain the usefulness of clinical algorithms. It is also important to promote research on promising tools such as automated reminders6 and critiquing prescription decisions (automatically criticising inappropriate prescriptions).37 Be cautious and don't throw the baby out with the bathwater!

Acknowledgements

The author would like to thank Dr CC Apfel and Katherine Fero from the Perioperative Clinical Research Core, Department of Anesthesia & Perioperative Care, UCSF Medical Center at Mt. Zion, San Francisco, for their thoughtful comments and editorial assistance.

References

1 Macario A, Weinger M, Carney S, Kim A. Which clinical anesthesia outcomes are important to avoid? The perspective of patients. Anesth Analg 1999; 89:652–658.
2 Gan TJ, Meyer TA, Apfel CC, et al. Society for Ambulatory Anesthesia guidelines for the management of postoperative nausea and vomiting. Anesth Analg 2007; 105:1615–1628.
3 Diemunsch P, Société Française d'Anesthésie et de Réanimation. Conference of experts: short text. Management of postoperative nausea and vomiting. French Society of Anesthesia and Resuscitation. Ann Fr Anesth Reanim 2008; 27:866–878.
4 Apfel CC, Korttila K, Abdalla M, et al. A factorial trial of six interventions for the prevention of postoperative nausea and vomiting. N Engl J Med 2004; 350:2441–2451.
5 Carlisle J, Stevenson C. Drugs for preventing postoperative nausea and vomiting. Cochrane Database Systematic Rev 2006:CD004125. doi: 10.1002/14651858.CD004125.pub2.
6 Kooij FO, Klok T, Hollmann MW, Kal JE. Automated reminders increase adherence to guidelines for administration of prophylaxis for postoperative nausea and vomiting. Eur J Anaesthesiol 2010; 27:187–191.
7 Franck M, Radtke FM, Baumeyer A, et al. Adherence to treatment guidelines for postoperative nausea and vomiting. How well does knowledge transfer result in improved clinical care? Anaesthesist 2010; 59:524–528.
8 Scuderi PE. PRO: anatomical classification of surgical procedures improves our understanding of the mechanisms of postoperative nausea and vomiting. Anesth Analg 2010; 110:410–411.
9 Kranke P, Eberhart LH, Gan TJ, et al. Algorithms for the prevention of postoperative nausea and vomiting: an efficacy and efficiency simulation. Eur J Anaesthesiol 2007; 24:856–867.
10 Apfel CC, Kranke P, Eberhart LHJ, et al. Comparison of predictive models for postoperative nausea and vomiting. Br J Anaesth 2002; 88:234–240.
11 Eberhart LHJ, Geldner G, Kranke P, et al. The development and validation of a risk score to predict the probability of postoperative vomiting in pediatric patients. Anesth Analg 2004; 99:1630–1637.
12 Kranke P, Eberhart LH, Toker H, et al. A prospective evaluation of the POVOC score for the prediction of postoperative vomiting in children. Anesth Analg 2007; 105:1592–1597.
13 Apfel CC, Stoecklein K, Lipfert P. PONV: a problem of inhalational anaesthesia? Best Pract Res Clin Anaesthesiol 2005; 19:485–500.
14 Ruiz JR, Kee SS, Frenzel JC, et al. The effect of an anatomically classified procedure on antiemetic administration in the postanesthesia care unit. Anesth Analg 2010; 110:403–409.
15 Grover SA, Lowensteyn I, Esrey KL, et al. Do doctors accurately assess coronary risk in their patients? Preliminary results of the coronary health assessment study. BMJ 1995; 310:975–978.
16 Koivuranta M, Laara E, Snare L, Alahuhta S. A survey of postoperative nausea and vomiting. Anaesthesia 1997; 52:443–449.
17 Apfel CC, Laara E, Koivuranta M, et al. A simplified risk score for predicting postoperative nausea and vomiting: conclusions from cross-validations between two centers. Anesthesiology 1999; 91:693–700.
18 Vermeersch P, Bossuyt X. Comparative analysis of different approaches to report diagnostic accuracy. Arch Intern Med 2010; 170:734–735.
19 Pierre S, Benais H, Pouymayou J. Apfel's simplified score may favourably predict the risk of postoperative nausea and vomiting. Can J Anaesth 2002; 49:237–242.
20 Pepe MS, Janes H, Longton G, et al. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 2004; 159:882–890.
21 Wald NJ, Morris JK. Assessing risk factors as potential screening tests: a simple assessment tool. Arch Intern Med. Published online October 25, 2010. doi:10.1001/archinternmed.2010.378 [Epub ahead of print]
22 Apfel CC, Kranke P, Greim CA, Roewer N. What can be expected from risk scores for predicting postoperative nausea and vomiting? Br J Anaesth 2001; 86:822–827.
23 Vickers AJ, Cronin AM. Traditional statistical methods for evaluating prediction models are uninformative as to clinical value: towards a decision analytic framework. Semin Oncol 2010; 37:31–38.
24 van den Bosch JE, Kalkman CJ, Vergouwe Y, et al. Assessing the applicability of scoring systems for predicting postoperative nausea and vomiting. Anaesthesia 2005; 60:323–331.
25 Engel JM, Junger A, Hartmann B, et al. Performance and customization of 4 prognostic models for postoperative onset of nausea and vomiting in ear, nose, and throat surgery. J Clin Anesth 2006; 18:256–263.
26 Sigaut S, Merckx P, Peuch C, et al. Does an educational strategy based on systematic preoperative assessment of simplified Apfel's score decrease postoperative nausea and vomiting? Ann Fr Anesth Reanim 2010; 29:765–769
27 McGinn TG, Guyatt GH, Wyer PC, et al. Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. JAMA 2000; 284:79–84.
28 Pierre S, Corno G, Benais H, Apfel CC. A risk score-dependent antiemetic approach effectively reduces postoperative nausea and vomiting: a continuous quality improvement initiative. Can J Anaesth 2004; 51:320–325.
29 Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006; 144:201–209.
30 Biedler A, Wermelt J, Kunitz O, et al. A risk adapted approach reduces the overall institutional incidence of postoperative nausea and vomiting. Can J Anaesth 2004; 5:13–19.
31 Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999; 282:1458–1465.
32 Scuderi PE. You (still) can't disprove the existence of dragons. Anesthesiology 2005; 102:1081–1082.
33 Rappaport BA. FDA response to droperidol black box warning editorials. Anesth Analg 2008; 106:1585.
34 McKechnie K, Froese A. Ventricular tachycardia after ondansetron administration in a child with undiagnosed long QT syndrome. Can J Anaesth 2010; 57:453–457.
35 Glavin RJ. Drug errors: consequences, mechanisms, and avoidance. Br J Anaesth 2010; 105:76–82.
36 Loftus RW, Koff MD, Burchman CC, et al. Transmission of pathogenic bacterial organisms in the anesthesia work area. Anesthesiology 2008; 109:399–407.
37 Lamy J-B, Ebrahiminia V, Riou C, et al. How to translate therapeutic recommendations in clinical practice guidelines into rules for critiquing physician prescriptions? Methods and application to five guidelines. BMC Med Inform Decis Mak 2010; 10:31.
Keywords:

algorithm; decision support systems; guideline; guideline adherence; post-operative nausea and vomiting; receiver operating characteristic curve; risk assessment

© 2011 European Society of Anaesthesiology