This Invited Commentary accompanies the following article:
Jammer Ib, Wickboldt N, Sander M, et al. Standards for definitions and use of outcome measures for clinical effectiveness research in perioperative medicine: European Perioperative Clinical Outcome (EPCO) definitions. A statement from the ESA-ESICM joint taskforce on perioperative outcome measures. Eur J Anaesthesiol 2015; 32:88–105.
‘In God we trust; all others must bring data’.
W. Edward Deming (1900–1993)
‘Le bon Dieu est dans le détail’ (the good God is in the detail)
Gustave Flaubert (1821–1880)
Which measures best define outcome following major surgery? The answer to this apparently dull and esoteric question has major implications for any clinician passionate about improving perioperative patient care. Clinical decisions should be based on the best available information. The highest standard of evidence in clinical medicine is a systematic review of high-quality randomised controlled trials (RCTs) with limited heterogeneity. Although some areas of clinical practice are blessed with a large number of homogeneous high-quality RCTs, other areas are less well served. Perioperative care is one such area; heterogeneity is common and is a substantial threat to the effective application of high-quality clinical evidence for patient benefit. Heterogeneity undermines evidenced-based medicine by limiting our capacity to compare, contrast and combine results from different studies. Specifically, heterogeneity may preclude the pooling of data from different papers in a meta-analysis, thereby robbing clinicians of the opportunity to benefit from the combined power of studies addressing the same clinical question. Alternatively, where such an analysis is undertaken, study heterogeneity reduces confidence in the direction and magnitude of a calculated treatment effect.1
The analogy of comparing apples and pears is commonly used to highlight the limitations inherent in the analysis of heterogeneous data. Clinical heterogeneity (as distinct from methodological or statistical heterogeneity) is often thought of in terms of settings, participants and interventions, but variation in outcome reporting is an important example. Differences between studies in the way outcomes are defined and when they are reported may lead to differences in the observed intervention effects.1 Although reporting of primary outcome variables may be similar in most or all studies in a systematic review, analysis of secondary outcomes is frequently limited by marked heterogeneity of criteria and timing. A recent Cochrane systematic review of perioperative haemodynamic management2 included 31 eligible studies and reported that ‘No two studies used the same list of morbidities after surgery. In most cases, no specific criteria were listed for morbidities. No two studies used the same criteria’. The authors concluded that ‘Diverse criteria and descriptions for morbidities, along with infrequent use of validated metrics, limited the precision of treatment effect estimates and the confidence that can be attached to them. Furthermore, pooling of different types of morbidity was inconsistent, limiting assessment of the overall “morbidity load”’.
The publication of ‘Standards for definitions and use of outcome measures for clinical effectiveness research in perioperative medicine: European Perioperative Clinical Outcome (EPCO) definitions’ in this issue of the European Journal of Anaesthesiology is a welcome step towards standardisation of outcome reporting in the perioperative literature.3 A joint taskforce of the European Society of Intensive Care Medicine (ESICM) and the European Society of Anaesthesiology (ESA) led by Professor Rupert Pearse (UK) reports the results of a structured process leading to the identification of a list of suggested outcomes for perioperative studies along with a proposed severity grading system. The standards include specific definitions of adverse events, composite outcomes describing short-term postoperative harm and quality of recovery, and health-related quality of life measures. The focus on duration of follow-up for outcomes reporting is welcome and reflects increasing recognition of the long-term implications of events occurring around the time of surgery.4,5 The group recommends that mortality data be reported at a minimum of 90 days following surgery and ideally at 1 year. Definitions for individual adverse events derive from a variety of sources. In some cases, definitions are based on ‘industry standard’ criteria used across many specialties, for example the USA Center for Disease Control (CDC) definitions for infection. For other outcomes, including postoperative haemorrhage and stroke, the group adopts definitions used by the well established American College of Surgeons - National Surgical Quality Improvement Programme (ACS-NSQIP) in the United States (http://site.acsnsqip.org/wp-content/uploads/2012/03/ACS-NSQIP-Participant-User-Data-File-User-Guide_06.pdf [Accessed 22 July 2014]). For a handful of outcomes, arbitrary definitions with no referenced source are adopted (e.g. deep vein thrombosis, cardiogenic pulmonary oedema). This reflects current inconsistency, and in some cases controversy, around the criteria defining these phenomena. There are some inevitable inconsistencies. The definition of acute myocardial infarction included in the composite measure of Major Adverse Cardiac Events (MACE) is not the same as the criteria suggested when this diagnosis is listed alone. The inclusion of the recently suggested definition for Myocardial Injury after Non-Cardiac Surgery (MINS) is interesting and may contribute to the wider adoption of this construct.6 Notwithstanding these minor issues, this publication is a landmark for Health Services Research in anaesthesia and an important step in the development of the specialty of perioperative medicine.7
Looking forward, this document will be a valuable input into the Core Outcome Measures in Effectiveness Trials (COMET) initiative for perioperative and anaesthetic care (http://www.niaa-hsrc.org.uk/COMPAC). The aim of the COMET process is to define a core outcome set for all studies to enable reliable comparison and combination of trial data.8 Importantly, this allows investigators to increase the value of their studies whilst placing no restriction on the breadth of outcomes reported by particular studies for specific purposes. The COMET process involves the establishment of a stakeholder group incorporating patients, carers and health professionals, including doctors. A protocol is developed and agreed in a manner analogous to that used for high-quality systematic reviews. The three key steps are a comprehensive and systematic approach to identifying a ‘long-list’ of relevant outcomes, a Delphi process involving all stakeholders to select a ‘short-list’ of candidate outcomes and a final consensus process to agree the core outcome set.
The ESICM and ESA should be congratulated for having the foresight to support the Joint Taskforce on Clinical Outcome Measures. The standards the Taskforce have produced highlight the importance of consistency of outcomes reporting between studies and provide a useful and up-to-date summary of many of the commonly used measures. This is an important step on the journey towards standardisation of reporting of perioperative outcomes. Such progress improves our capacity to compare, contrast and combine data from clinical effectiveness studies. The benefit for clinicians and patients will be better information on which to base important clinical decisions and improved care for patients undergoing surgery.
Acknowledgements relating to this article
Assistance with the Commentary: none.
Financial support and sponsorship: The author is paid by the Royal College of Anaesthetists (UK) for his role as director of the UK National Institute of Academic Anaesthesia (NIAA) Health Services Research Centre (HSRC).
Conflicts of interest: The author is director of the UK National Institute of Academic Anaesthesia (NIAA) Health Services Research Centre (HSRC) and chair of COMPAC (Core Outcome Measures in Perioperative and Anaesthetic Care). He also serves on the board of the UK Faculty of Intensive Care Medicine.
Comment from the Editor: this Invited Commentary was checked and accepted by the editors but was not sent for external peer review.
1. Gagnier JJ, Morgenstern H, Altman DG, et al. Consensus-based recommendations for investigating clinical heterogeneity in systematic reviews. BMC Med Res Methodol
2. Grocott MP, Dushianthan A, Hamilton MA, et al. Perioperative increase in global blood flow to explicit defined goals and outcomes after surgery: a Cochrane systematic review. Br J Anaesth
3. Jammer Ib, Wickboldt N, Sander M, et al. Standards for definitions and use of outcome measures for clinical effectiveness research in perioperative medicine: European Perioperative Clinical Outcome (EPCO) definitions. A statement from the ESA-ESICM joint taskforce on perioperative outcome measures. Eur J Anaesthesiol
4. Khuri SF, Henderson WG, DePalma RG, et al. Determinants of long-term survival after major surgery and the adverse effect of postoperative complications. Ann Surg
5. Moonesinghe SR, Harris S, Mythen MG, et al. Survival after postoperative morbidity: a longitudinal observational cohort study. Br J Anaesth
2014; [Epub ahead of print].
6. Botto F, Alonso-Coello P, Chan MT, et al. Myocardial injury after noncardiac surgery: a large, international, prospective cohort study establishing diagnostic criteria, characteristics, predictors, and 30-day outcomes. Anesthesiology
7. Grocott MP, Pearse RM. Perioperative medicine: the future of anaesthesia? Br J Anaesth
8. Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One