Among the 86 studies included in this review, 52.9% were from North America, 82.4% were academia driven, and 80.2% investigated a postoperative event (Table 1). They were mainly explanatory with a tested hypothesis (68.6%) and based on logistic regression (80.2%). Bibliographic references of the studies included are presented in Appendix 2, and their individual description in Appendices 3–5. Slight but not striking differences were observed among the 4 journals included in this review (Appendix 6).
Reporting and STROBE Checklist
Among the 21 items of reporting common to both predictive and explanatory studies, the median reporting rate was 79% (27th−75th percentiles, 36–91) (Table 2). It was 100% (82–100) in predictive studies, 75% (44–87) in explanatory studies without tested hypothesis, and 76% (25–90) in explanatory studies with tested hypothesis.
Six items had a reporting rate <36% (i.e., the 25th percentile), with some of them not identified with the STROBE checklist: evaluation of the outcome (11.9%), reason for sample size (15.1%), handling of missing data (36.0%), assessment of colinearity (17.4%), interactions (13.9%), and calibration (34.9%).
The results of the assessment of the included articles with the STROBE checklist are presented in Appendix 7. The reporting of the description of all statistical methods was considered present in 95.3% of the studies.
Assessment of the Risk of Bias in Individual Studies
The individual risk of bias was small except for the lack of blinded assessment of the primary outcome (88.1%), and to a lesser extent, for >1 outcome studied (46.5%) and the presence of spin (42.1%) (Table 3).
The selection of candidate variables for inclusion in the multivariable analysis based on statistical screening only was observed in 27.5% of the studies (Table 4), the ratio of the number of events to the number of candidate variables <10 in 44.6%, complete case analysis of missing data in 93.6%, and categorization of 1 or all continuous candidates in 65.1%.
This methodological descriptive review identified an acceptably good reporting of multivariable analysis with a median reporting rate of 79%. Some bias and methodological shortcomings, especially observed in explanatory studies, may hinder the reliability of the analysis but could be easily corrected.
Contrasting with this good reporting rate, some points were obviously underreported, such as evaluation of the outcome, reason for sample size, handling of missing data, and assessment of colinearity, interactions, and performance. Complete and accurate reporting of the multivariable analysis should avoid the bias of assessing the quality of studies based only on the quality of their reporting.16 Moreover, it should also allow the reader to assess the applicability of the study to his/her own clinical practice. Poor reporting may lead to inappropriate decisions and consequences.36 There is evidence in effectiveness research that demonstrates the consequences of bad reporting.37 The studies included in this review were too recent to have led to inappropriate recommendations based on bad reporting.
The ever-growing plethora of available reporting guidelines may confuse and discourage the authors (Consolidated Standards of Reporting Trials [CONSORT] for randomized controlled trials, STROBE for observational studies…).38 The EQUATOR network website provides a comprehensive and updated list of statements and checklists for reporting adapted to each study design (http://www.equator-network.org/). This website should be consulted by authors when they report the results of their study. It may also help young investigators design their own studies. However, authors are probably not the only ones to be accountable for underreporting. Reviewers, editors, and publishers share responsibility and must contribute to the improvement of the current reporting. For instance, Cobo et al.39 demonstrated that using reporting guidelines during the peer-review process increased the quality of final manuscripts submitted to a biomedical journal.
The results of the analysis of the studies with the STROBE checklist focusing on multivariable analysis differ on some items from the results of the analysis with our assessment form. With the STROBE checklist, the reporting of the description of all statistical methods was considered present in 95.3% of the included studies. With our analysis, some aspects of the multivariable analysis were clearly underreported. To date, Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) recommendations for the reporting of tumor marker prognostic studies and PROGnosis RESearch Strategy (PROGRESS) recommendations for prognosis research strategy are the most comprehensive and appropriate guidelines for the reporting of multivariable analyses.3–6,40 Reminders of recommendations for reporting of multivariable analysis based on STROBE, REMARK, and PROGRESS recommendations are presented in Table 5.
Some aspects of reporting recommended by these statements may, however, not be applicable to explanatory studies. For instance, one does not expect to see an internal or external validation in an explanatory study. However, the items of reporting, up to the stage of validation and clinical usefulness, are common to both types of studies. The reminder in Table 5 clearly differentiates the level of reporting for an explanatory and a predictive study. In addition, for explanatory studies with tested hypothesis, the only odds or hazard ratio of direct relevance is the one that assesses the hypothesized association between the risk factor and the outcome, adjusted for the other variables. Reporting and discussing odds or hazard ratio for all the adjusting variables used in this type of study may confuse the reader and increase the probability of spin.
Assessment of Bias in Individual Studies
The individual risk of bias was high for the blinded assessment of the primary outcome (detection bias), and, to a lesser extent, for the number of outcomes studied (reporting bias) and the presence of spin. If measurement of the outcome involves observer interpretation, it should be measured without knowledge of the candidate variables to avoid detection bias, especially in prospective studies. However, when the outcome is all-cause mortality (but not cause-specific mortality), blinding of the assessor is of lesser concern. In the current review, the measurement of the primary outcome was not reported as blinded in 88.1% of the studies in which the primary outcome was not death. When many outcomes are examined, especially when outcomes are not a priori defined, there is potential for bias in the selection of outcomes for multivariable analysis. In this review, 46.5% of the studies reported >1 outcome. Study registration and publication of the primary and secondary outcome measures and of the planned statistical analysis plan could prevent selective reporting and publication bias. Similar to controlled trials with statistically nonsignificant results for primary outcomes, 42.1% of explanatory studies with an a priori hypothesis and negative results for the hypothesis had spin.35
There is no consensus on the best method for selecting candidate explanatory variables entered in the multivariable analysis. Some methods are particularly cautioned against, such as inclusion or exclusion of variables based only on univariable analysis observed in 27.5% of the included studies. It can be important to retain candidate variables known to be important from the literature but that may not reach statistical significance in a particular dataset. It is recommended that candidate variables that have clinical relevance or are already well established in the literature should be retained in models.15 Another concern with candidate variables is colinearity, which refers to the fact that variables may be strongly correlated with each other.15 Colinearity hampers reliable estimation of regression coefficients of the correlated variables. In the current review, only 17.4% of the studies reported that colinearity among candidate variables was assessed. The mathematical formulation of logistic regression or Cox models implies an additive effect of the variables on the outcome considered. However, the effect of a variable may depend on the effect of another one, a phenomenon called a 2-way interaction. In this case, the effect of 1 variable cannot be interpreted alone. In the current review, only 14% of the studies reported that interactions among variables were assessed.
Collecting all data on explanatory variables and outcomes for all patients rarely is achieved. A common approach to handle missing data is to exclude individuals with missing values and conduct a complete case analysis. However, this approach discards useful information and can lead to biased results.41 Imputation techniques are a valid approach to minimize the effect of missing data.15 In this review, information regarding missing data was reported in only 36% of the studies. Only 2 studies used imputation techniques. To assess the representativeness and quality of the data, reporting the number of missing data by variable and the number of individuals with complete data on all variables is advised.40
The predictive accuracy of a prediction model is determined by the number of events of the outcome (“effective sample size”) and not by the number of patients included in the study (“real sample size”).15 Although models in exploratory studies often have a more liberal approach than in predictive ones (“black box approach” versus “parsimonious approach” and restriction of predictors), the general recommendation is to have at least 10 events or even 20 of the studied outcome per explanatory variables. In this review, only 55.4% had a ratio >10 and 38.5% had >20. This “rule of thumb” should be considered when defining the study sample size.42,43 Ignoring this rule, especially in small sample size studies, leads to biased regression coefficients. In this review, in only 15.1% of the studies did authors report a justification for sample size based on the number of events or on the ratio of number of events to number of candidates.
The final model can be strongly affected by the coding of the variables, especially continuous variables.8,9,28 The practice of dichotomizing continuous candidate explanatory variables at the median value or at an arbitrary cutoff is not recommended, because it causes loss of information and statistical power. In addition, it results in unrealistic steps in the predicted risk, with patients at either side of a cutoff point categorized with very different levels of risk.44,45 In this review, categorization of a continuous variable was used in 65.1% of the studies. Continuous candidate variables should be kept continuous in the model. If the variable has a nonlinear relationship with the outcome, use of splines or fractional polynomial functions is recommended.40
Limits of the Study
The arbitrary choice to include the 4 general anesthesiology journals with the greatest impact factor may have biased the results. To the best of our knowledge, no study has specifically examined the relationship between impact factor and the quality of statistical methods in anesthesiology journals. Two studies indirectly indicate that the quality of statistical methods does not vary among anesthesiology journals, and we did not observe striking differences among the journals included in this review46,47 (Appendix 6).
Limiting this review to a 2-year period did not allow the assessment of temporal trends. However, the assessment of reporting and methodology of multivariable analysis in anesthesiology journals was never done before. This review should therefore be viewed as a first step in this assessment and as an incentive to redo it in a few years to determine whether the quality of reporting and methodology increases or decreases over time.
We deliberately chose not to contact authors because the aim of this study was not to assess the discrepancy between the reporting of the methodological aspects of the studies and what was really done by the authors.16 In other words, we wanted to assess what constitutes the available information for the common readers who have to analyze the study and make their own judgments. We considered that the reader of a study published in high impact factor journals such as the 4 journals included in this review should obtain the information required to assess the quality of the study without any additional effort. This point is all the more relevant because it is more and more difficult to keep up with the increasing amount of available information.48
We arbitrarily limited the method to consider confounding variables to statistical adjustment with multivariable analysis, because it is 1 of the most often used methods in medical literature. Other methods such as stratification, matching, or propensity scores can be used for this purpose, especially in explanatory studies.
The reporting of multivariable analysis in explanatory and predictive studies was acceptably good. Some points could be easily improved by the authors by checking reporting guidelines and the EQUATOR website and by editors by adopting editorial policies. Similarly, some methodological shortcomings could be easily corrected with limiting the number of candidate variables, including cases with missing data and not arbitrarily categorizing continuous variables.
Search Strings for the Identification of Studies
- On Medline database through Web of Knowledge
- ((((TS=statistical models OR TS=risk factors OR TS=propensity score OR TS=multivariable analysis OR TS=decision making OR TS=validation studies OR TS=prognosis OR TS=outcome OR TS=prediction model OR TS=logistic regression OR TS=cox model) AND (TS=anesthesia OR TS=anesthesiology OR TS=perioperative care)))) AND Language=(English)
- Refined by: Source Titles=(ANESTHESIA AND ANALGESIA OR ANESTHESIOLOGY OR BRITISH JOURNAL OF ANAESTHESIA OR ANAESTHESIA)
- In this syntax (e.g., TS=risk factors), risk factors (without quoting) corresponds to “risk factors” and not to “risk or factors or risk factors.” TS is short for topic field. TS=risk factors find records containing the term risk factors in the Abstract, Title, or Key words. This is not a full-text search.
- On PubMed
- (“statistical models” OR “ risk factors” OR “propensity score” OR “multivariable analysis” OR “decision making” OR “validation studies” OR “prognosis” OR “outcome” OR “prediction model” OR “logistic regression” OR “cox model”)
- AND (“anesthesia” OR “anesthesiology” OR “perioperative care”)
- AND (“Anesthesiology”[Journal] OR “british journal of anaesthesia”[Journal] OR “Anesthesia and analgesia”[Journal] OR “Anaesthesia”[Journal])
- AND (“2010/01/01”[PDAT]: “2011/12/31”[PDAT])
- On the websites of the anesthesia journals
Statistical models OR risk factors OR propensity score OR multivariate analysis OR decision making OR validation studies OR prognosis OR outcome OR prediction model OR logistic regression OR cox model
Bibliographic References of the 86 Studies Included in the Review
1. Aasvang EK, Gmaehle E, Hansen JB, Gmaehle B, Forman JL, Schwarz J, Bittner R, Kehlet H. Predictive risk factors for persistent postherniotomy pain. Anesthesiology 2010;112:957–69
2. Amar D, Munoz D, Shi W, Zhang H, Thaler HT. A clinical prediction rule for pulmonary complications after thoracic surgery for primary lung cancer. Anesth Analg 2010;110:1343–8
3. Argalious M, Xu M, Sun Z, Smedira N, Koch CG. Preoperative statin therapy is not associated with a reduced incidence of postoperative acute kidney injury after cardiac surgery. Anesth Analg 2010;111:324–30
4. Aronson S, Dyke CM, Levy JH, Cheung AT, Lumb PD, Avery EG, Hu MY, Newman MF. Does perioperative systolic blood pressure variability predict mortality after cardiac surgery? An exploratory analysis of the ECLIPSE trials. Anesth Analg 2011;113:19–30
5. Aronson S, Stafford-Smith M, Phillips-Bute B, Shaw A, Gaca J, Newman M; Cardiothoracic Anesthesiology Research Endeavors. Intraoperative systolic blood pressure variability predicts 30-day mortality in aortocoronary bypass surgery patients. Anesthesiology 2010;113:305–12
6. Ausset S, Auroy Y, Verret C, Benhamou D, Vest P, Cirodde A, Lenoir B. Quality of postoperative care after major orthopedic surgery is correlated with both long-term cardiovascular outcome and troponin Ic elevation. Anesthesiology 2010;113:529–40
7. Aziz MF, Healy D, Kheterpal S, Fu RF, Dillman D, Brambrink AM. Routine clinical practice effectiveness of the Glidescope in difficult airway management: an analysis of 2,004 Glidescope intubations, complications, and failures from two institutions. Anesthesiology 2011;114:34–41
8. Bateman BT, Berman MF, Riley LE, Leffert LR. The epidemiology of postpartum hemorrhage in a large, nationwide sample of deliveries. Anesth Analg 2010;110:1368–73
9. Benzon HT, Avram MJ, Benzon HA, Kirby-Nolan M, Nader A. Factor VII levels and international normalized ratios in the early phase of warfarin therapy. Anesthesiology 2010;112:298–304
10. Besch G, Liu N, Samain E, Pericard C, Boichut N, Mercier M, Chazot T, Pili-Floury S. Occurrence of and risk factors for electroencephalogram burst suppression during propofol-remifentanil anaesthesia. Br J Anaesth 2011;107:749–56
11. Boutonnet M, Faitot V, Katz A, Salomon L, Keita H. Mallampati class changes during pregnancy, labour, and after delivery: can these be predicted? Br J Anaesth 2010;104:67–70
12. Brandom BW, Larach MG, Chen MS, Young MC. Complications associated with the administration of dantrolene 1987 to 2006: a report from the North American Malignant Hyperthermia Registry of the Malignant Hyperthermia Association of the United States. Anesth Analg 2011;112:1115–23
13. Canet J, Gallart L, Gomar C, Paluzie G, Vallès J, Castillo J, Sabaté S, Mazo V, Briones Z, Sanchis J; ARISCAT Group. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology 2010;113:1338–50
14. Chang CC, Lin HC, Lin HW, Lin HC. Anesthetic management and surgical site infections in total hip or knee replacement: a population-based study. Anesthesiology 2010;113:279–84
15. Collyer TC, Reynolds HC, Truyens E, Kilshaw L, Corcoran T. Perioperative management of clopidogrel therapy: the effects on in-hospital cardiac morbidity in older patients with hip fractures. Br J Anaesth 2011;107:911–5
16. Dalton JE, Kurz A, Turan A, Mascha EJ, Sessler DI, Saager L. Development and validation of a risk quantification index for 30-day postoperative mortality and morbidity in noncardiac surgical patients. Anesthesiology 2011;114:1336–44
17. Davidson AJ, Smith KR, Blussé van Oud-Alblas HJ, Lopez U, Malviya S, Bannister CF, Galinkin JL, Habre W, Ironfield C, Voepel-Lewis T, Weber F. Awareness in children: a secondary analysis of five cohort studies. Anaesthesia 2011;66:446–54
18. Deiner SG, Kwatra SG, Lin HM, Weisz DJ. Patient characteristics and anesthetic technique are additive but not synergistic predictors of successful motor evoked potential monitoring. Anesth Analg 2010;111:421–5
19. DiMaggio C, Sun LS, Li G. Early childhood exposure to anesthesia and risk of developmental and behavioral disorders in a sibling birth cohort. Anesth Analg 2011;113:1143–51
20. Duncan AE, Abd-Elsayed A, Maheshwari A, Xu M, Soltesz E, Koch CG. Role of intraoperative and postoperative blood glucose concentrations in predicting outcomes after cardiac surgery. Anesthesiology 2010;112:860–71
21. Durga P, Sahu BP, Mantha S, Ramachandran G. Development and validation of predictors of respiratory insufficiency and mortality scores: simple bedside additive scores for prediction of ventilation and in-hospital mortality in acute cervical spine injury. Anesth Analg 2010;110:134–40
22. Fernandez R, Tubau I, Masip J, Muñoz L, Roig I, Artigas A. Low reticulocyte hemoglobin content is associated with a higher blood transfusion rate in critically ill patients: a cohort study. Anesthesiology 2010;112:1211–5
23. Flick RP, Lee K, Hofer RE, Beinborn CW, Hambel EM, Klein MK, Gunn PW, Wilder RT, Katusic SK, Schroeder DR, Warner DO, Sprung J. Neuraxial labor analgesia for vaginal delivery and its effects on childhood learning disabilities. Anesth Analg 2011;112:1424–31
24. Flu WJ, van Kuijk JP, Hoeks SE, Kuiper R, Schouten O, Goei D, Elhendy A, Verhagen HJ, Thomson IR, Bax JJ, Fleisher LA, Poldermans D. Prognostic implications of asymptomatic left ventricular dysfunction in patients undergoing vascular surgery. Anesthesiology 2010;112:1316–24
25. Forget P, Vandenhende J, Berliere M, Machiels JP, Nussbaum B, Legrand C, De Kock M. Do intraoperative analgesics influence breast cancer recurrence after mastectomy? A retrospective analysis. Anesth Analg 2010;110:1630–5
26. Fox AA, Muehlschlegel JD, Body SC, Shernan SK, Liu KY, Perry TE, Aranki SF, Cook EF, Marcantonio ER, Collard CD. Comparison of the utility of preoperative versus postoperative B-type natriuretic peptide for predicting hospital length of stay and mortality after primary coronary artery bypass grafting. Anesthesiology 2010;112:842–51
27. Garvin S, Muehlschlegel JD, Perry TE, Chen J, Liu KY, Fox AA, Collard CD, Aranki SF, Shernan SK, Body SC. Postoperative activity, but not preoperative activity, of antithrombin is associated with major adverse cardiac events after coronary artery bypass graft surgery. Anesth Analg 2010;111:862–9
28. Glance LG, Wissler R, Mukamel DB, Li Y, Diachun CA, Salloum R, Fleming FJ, Dick AW. Perioperative outcomes among patients with the modified metabolic syndrome who are undergoing noncardiac surgery. Anesthesiology 2010;113:859–72
29. Glance LG, Dick AW, Mukamel DB, Fleming FJ, Zollo RA, Wissler R, Salloum R, Meredith UW, Osler TM. Association between intraoperative blood transfusion and mortality and morbidity in patients undergoing noncardiac surgery. Anesthesiology 2011;114:283–92
30. Gottschalk A, Ford JG, Regelin CC, You J, Mascha EJ, Sessler DI, Durieux ME, Nemergut EC. Association between epidural analgesia and cancer recurrence after colorectal cancer surgery. Anesthesiology 2010;113:27–34
31. Gupta A, Björnsson A, Fredriksson M, Hallböök O, Eintrei C. Reduction in mortality after epidural anaesthesia and analgesia in patients undergoing rectal but not colonic cancer surgery: a retrospective analysis of data from 655 patients in central Sweden. Br J Anaesth 2011;107:164–70
32. Haller G, Courvoisier DS, Anderson H, Myles PS. Clinical factors associated with the non-utilization of an anaesthesia incident reporting system. Br J Anaesth 2011;107:171–9
33. Hansen TG, Pedersen JK, Henneberg SW, Pedersen DA, Murray JC, Morton NS, Christensen K. Academic performance in adolescence after inguinal hernia repair in infancy: a nationwide cohort study. Anesthesiology 2011;114:1076–85
34. Heringlake M, Garbers C, Käbler JH, Anderson I, Heinze H, Schön J, Berger KU, Dibbelt L, Sievers HH, Hanke T. Preoperative cerebral oxygen saturation and clinical outcomes in cardiac surgery. Anesthesiology 2011;114:58–69
35. Hindman BJ, Bayman EO, Pfisterer WK, Torner JC, Todd MM; IHAST Investigators. No association between intraoperative hypothermia or supplemental protective drug and neurologic outcomes in patients undergoing temporary clipping during cerebral aneurysm surgery: findings from the Intraoperative Hypothermia for Aneurysm Surgery Trial. Anesthesiology 2010;112:86–101
36. Holmberg TJ, Bowman SM, Warner KJ, Vavilala MS, Bulger EM, Copass MK, Sharar SR. The association between obesity and difficult prehospital tracheal intubation. Anesth Analg 2011;112:1132–8
37. Hughes CG, Weavind L, Banerjee A, Mercaldo ND, Schildcrout JS, Pandharipande PP. Intraoperative risk factors for acute respiratory distress syndrome in critically ill patients. Anesth Analg 2010;111:464–7
38. Ismail H, Ho KM, Narayan K, Kondalsamy-Chennakesavan S. Effect of neuraxial anaesthesia on tumour progression in cervical cancer patients treated with brachytherapy: a retrospective cohort study. Br J Anaesth 2010;105:145–9
39. Jacob AK, Mantilla CB, Sviggum HP, Schroeder DR, Pagnano MW, Hebl JR. Perioperative nerve injury after total hip arthroplasty: regional anesthesia risk during a 20-year cohort study. Anesthesiology 2011;115:1172–8
40. Jankowski CJ, Trenerry MR, Cook DJ, Buenvenida SL, Stevens SR, Schroeder DR, Warner DO. Cognitive and functional predictors and sequelae of postoperative delirium in elderly patients undergoing elective joint arthroplasty. Anesth Analg 2011;112:1186–93
41. Jun NH, Shim JK, Kim JC, Kwak YL. Prognostic value of a tissue Doppler-derived index of left ventricular filling pressure on composite morbidity after off-pump coronary artery bypass surgery. Br J Anaesth 2011;107:519–24
42. Kertai MD, Pal N, Palanca BJ, Lin N, Searleman SA, Zhang L, Burnside BA, Finkel KJ, Avidan MS; B-Unaware Study Group. Association of perioperative risk factors and cumulative duration of low bispectral index with intermediate-term mortality after cardiac surgery in the B-Unaware Trial. Anesthesiology 2010;112:1116–27
43. Kertai MD, Palanca BJ, Pal N, Burnside BA, Zhang L, Sadiq F, Finkel KJ, Avidan MS; B-Unaware Study Group. Bispectral index monitoring, duration of bispectral index below 45, patient risk factors, and intermediate-term mortality after noncardiac surgery in the B-Unaware Trial. Anesthesiology 2011;114:545–56
44. Kim WH, Ahn HJ, Lee CJ, Shin BS, Ko JS, Choi SJ, Ryu SA. Neck circumference to thyromental distance ratio: a new predictor of difficult intubation in obese patients. Br J Anaesth 2011;106:743–8
45. Kin N, Weismann C, Srivastava S, Chakravarti S, Bodian C, Hossain S, Krol M, Hollinger I, Nguyen K, Mittnacht AJ. Factors affecting the decision to defer endotracheal extubation after surgery for congenital heart disease: a prospective observational study. Anesth Analg 2011;113:329–35
46. Kor DJ, Warner DO, Alsara A, Fernández-Pérez ER, Malinchoc M, Kashyap R, Li G, Gajic O. Derivation and diagnostic accuracy of the surgical lung injury prediction model. Anesthesiology 2011;115:117–28
47. Kraemer FW, Stricker PA, Gurnaney HG, McClung H, Meador MR, Sussman E, Burgess BJ, Ciampa B, Mendelsohn J, Rehman MA, Watcha MF. Bradycardia during induction of anesthesia with sevoflurane in children with Down syndrome. Anesth Analg 2010;111:1259–63
48. Le Manach Y, Ibanez Esteves C, Bertrand M, Goarin JP, Fléron MH, Coriat P, Koskas F, Riou B, Landais P. Impact of preoperative statin therapy on adverse postoperative outcomes in patients undergoing vascular surgery. Anesthesiology 2011;114:98–104
49. Leslie K, Myles PS, Forbes A, Chan MT. The effect of bispectral index monitoring on long-term survival in the B-aware trial. Anesth Analg 2010;110:816–22
50. Leslie K, Myles PS, Chan MT, Forbes A, Paech MJ, Peyton P, Silbert BS, Williamson E. Nitrous oxide and long-term morbidity and mortality in the ENIGMA trial. Anesth Analg 2011;112:387–93
51. Lin L, Liu C, Tan H, Ouyang H, Zhang Y, Zeng W. Anaesthetic technique may affect prognosis for ovarian serous adenocarcinoma: a retrospective analysis. Br J Anaesth 2011;106:814–22
52. Lindholm ML, Granath F, Eriksson LI, Sandin R. Malignant disease within 5 years after surgery in relation to duration of sevoflurane anesthesia and time with bispectral index under 45. Anesth Analg 2011;113:778–83
53. Lobo SM, Rezende E, Knibel MF, Silva NB, Páramo JA, Nácul FE, Mendes CL, Assunção M, Costa RC, Grion CC, Pinto SF, Mello PM, Maia MO, Duarte PA, Gutierrez F, Silva JM Jr, Lopes MR, Cordeiro JA, Mellot C. Early determinants of death due to multiple organ failure after noncardiac surgery in high-risk patients. Anesth Analg 2011;112:877–83
54. Martin LD, Mhyre JM, Shanks AM, Tremper KK, Kheterpal S. 3,423 emergency tracheal intubations at a university hospital: airway outcomes and complications. Anesthesiology 2011;114:42–8
55. Mashour GA, Shanks AM, Kheterpal S. Perioperative stroke and associated mortality after noncardiac, nonneurologic surgery. Anesthesiology 2011;114:1289–96
56. Mauermann WJ, Nuttall GA, Cook DJ, Hanson AC, Schroeder DR, Oliver WC. Hemofiltration during cardiopulmonary bypass does not decrease the incidence of atrial fibrillation after cardiac surgery. Anesth Analg 2010;110:329–34
57. McDonagh DL, Mathew JP, White WD, Phillips-Bute B, Laskowitz DT, Podgoreanu MV, Newman MF; Neurologic Outcome Research Group. Cognitive function after major noncardiac surgery, apolipoprotein E4 genotype, and biomarkers of brain injury. Anesthesiology 2010;112:852–9
58. McKenny M, Ryan T, Tate H, Graham B, Young VK, Dowd N. Age of transfused blood is not associated with increased postoperative adverse outcome after cardiac surgery. Br J Anaesth 2011;106:643–9
59. Meier PM, Goobie SM, DiNardo JA, Proctor MR, Zurakowski D, Soriano SG. Endoscopic strip craniectomy in early infancy: the initial five years of anesthesia experience. Anesth Analg 2011;112:407–14
60. Memtsoudis SG, Ma Y, Chiu YL, Walz JM, Voswinckel R, Mazumdar M. Perioperative mortality in patients with pulmonary hypertension undergoing major joint replacement. Anesth Analg 2010;111:1110–6
61. Memtsoudis SG, Ma Y, Chiu YL, Poultsides L, Gonzalez Della Valle A, Mazumdar M. Bilateral total knee arthroplasty: risk factors for major morbidity and mortality. Anesth Analg 2011;113:784–90
62. Mhyre JM, Ramachandran SK, Kheterpal S, Morris M, Chan PS; American Heart Association National Registry for Cardiopulmonary Resuscitation Investigators. Delayed time to defibrillation after intraoperative and periprocedural cardiac arrest. Anesthesiology 2010;113:782–93
63. Mhyre JM, Bateman BT, Leffert LR. Influence of patient comorbidities on the risk of near-miss maternal morbidity or mortality. Anesthesiology 2011;115:963–72
64. Moganasundram S, Hunt BJ, Sykes K, Holton F, Parmar K, Durward A, Murdoch IA, Austin C, Anderson D, Tibby SM. The relationship among thromboelastography, hemostatic variables, and bleeding after cardiopulmonary bypass surgery in children. Anesth Analg 2010;110:995–1002
65. Muehlschlegel JD, Perry TE, Liu KY, Fox AA, Collard CD, Shernan SK, Body SC. Heart-type fatty acid binding protein is an independent predictor of death and ventricular dysfunction after coronary artery bypass graft surgery. Anesth Analg 2010;111:1101–9
66. Nafiu OO, Kheterpal S, Moulding R, Picton P, Tremper KK, Campbell DA Jr, Eliason JL, Stanley JC. The association of body mass index to postoperative outcomes in elderly vascular surgery patients: a reverse J-curve phenomenon. Anesth Analg 2011;112:23–9
67. Nikolov NM, Fontes ML, White WD, Aronson S, Bar-Yosef S, Gaca JG, Podgoreanu MV, Stafford-Smith M, Newman MF, Mathew JP. Pulse pressure and long-term survival after coronary artery bypass graft surgery. Anesth Analg 2010;110:335–40
68. Noordzij PG, Poldermans D, Schouten O, Bax JJ, Schreiner FA, Boersma E. Postoperative mortality in The Netherlands: a population-based analysis of surgery-specific risk in adults. Anesthesiology 2010;112:1105–15
69. Powell ES, Cook D, Pearce AC, Davies P, Bowler GM, Naidu B, Gao F; UKPOS Investigators. A prospective, multicentre, observational cohort study of analgesia and outcome after pneumonectomy. Br J Anaesth 2011;106:364–70
70. Ramachandran SK, Kheterpal S, Consens F, Shanks A, Doherty TM, Morris M, Tremper KK. Derivation and validation of a simple perioperative sleep apnea prediction score. Anesth Analg 2010;110:1007–15
71. Ramachandran SK, Nafiu OO, Ghaferi A, Tremper KK, Shanks A, Kheterpal S. Independent predictors and outcomes of unanticipated early postoperative tracheal intubation after nonemergent, noncardiac surgery. Anesthesiology 2011;115:44–53
72. Ruiz JR, Kee SS, Frenzel JC, Ensor JE, Selvan M, Riedel BJ, Apfel C. The effect of an anatomically classified procedure on antiemetic administration in the postanesthesia care unit. Anesth Analg 2010;110:403–9
73. Sabaté S, Mases A, Guilera N, Canet J, Castillo J, Orrego C, Sabaté A, Fita G, Parramón F, Paniagua P, Rodríguez A, Sabaté M; ANESCARDIOCAT Group. Incidence and predictors of major perioperative adverse cardiac and cerebrovascular events in non-cardiac surgery. Br J Anaesth 2011;107:879–90
74. Schwann NM, Hillel Z, Hoeft A, Barash P, Möhnle P, Miao Y, Mangano DT. Lack of effectiveness of the pulmonary artery catheter in cardiac surgery. Anesth Analg 2011;113:994–1002
75. Sessler DI, Kurz A, Saager L, Dalton JE. Operation timing and 30-day mortality after elective general surgery. Anesth Analg 2011;113:1423–8
76. Shi Y, Warner DO. Pediatric surgery and parental smoking behavior. Anesthesiology 2011;115:12–7
77. Song JG, Jeong SM, Shin WJ, Jun IG, Shin K, Huh IY, Kim YK, Hwang GS. Laboratory variables associated with low near-infrared cerebral oxygen saturation in icteric patients before liver transplantation surgery. Anesth Analg 2011;112:1347–52
78. Tsai PS, Hsu CS, Fan YC, Huang CJ. General anaesthesia is associated with increased risk of surgical site infection after Caesarean delivery compared with neuraxial anaesthesia: a population-based study. Br J Anaesth 2011;107:757–61
79. van Lier F, van der Geest PJ, Hoeks SE, van Gestel YR, Hol JW, Sin DD, Stolker RJ, Poldermans D. Epidural analgesia is associated with improved health outcomes of surgical patients with chronic obstructive pulmonary disease. Anesthesiology 2011;115:315–21
80. Wallace AW, Au S, Cason BA. Association of the pattern of use of perioperative β-blockade and postoperative mortality. Anesthesiology 2010;113:794–805
81. Wallace AW, Au S, Cason BA. Perioperative β-blockade: atenolol is associated with reduced mortality when compared to metoprolol. Anesthesiology 2011;114:824–36
82. Wang JF, Bian JJ, Wan XJ, Zhu KM, Sun ZZ, Lu AD. NFKB1-94ins/del polymorphism is not associated with lung injury after cardiopulmonary bypass. Anaesthesia 2010;65:158–62
83. Weingarten TN, Flores AS, McKenzie JA, Nguyen LT, Robinson WB, Kinney TM, Siems BT, Wenzel PJ, Sarr MG, Marienau MS, Schroeder DR, Olson EJ, Morgenthaler TI, Warner DO, Sprung J. Obstructive sleep apnoea and perioperative complications in bariatric patients. Br J Anaesth 2011;106:131–9
84. Williams TA, Ho KM, Dobb GJ, Finn JC, Knuiman M, Webb SA; Royal Perth Hospital ICU Data Linkage Group. Effect of length of stay in intensive care unit on hospital and long-term mortality of critically ill adult patients. Br J Anaesth 2010;104:459–64
85. Wuethrich PY, Hsu Schmitz SF, Kessler TM, Thalmann GN, Studer UE, Stueber F, Burkhard FC. Potential influence of the anesthetic technique used during open radical prostatectomy on prostate cancer-related outcome: a retrospective study. Anesthesiology 2010;113:570–6
86. Yoo HS, Nahm FS, Lee PB, Lee CJ. Early thoracic sympathetic block improves the treatment effect for upper extremity neuropathic pain. Anesth Analg 2011;113:605–9
Appendix 3. Individual Results: General Characteristics and Primary Outcome
Appendix 4. Individual Results: Model Development
Appendix 5. Individual Results: Model Performance and Spin
Appendix 6. Comparison Among 3 of the 4 Journals Included in the Review on Some Points of Reporting and Methodology
Appendix 7. Assessment of the Included Articles in the Review with the STROBE Checklist for 4 Items Addressing the Reporting of Multivariable Analysis
Name: Jean Guglielminotti, MD, PhD.
Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.
Attestation: Jean Guglielminotti has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.
Name: Agnès Dechartres, MD, PhD.
Contribution: This author helped write the manuscript.
Attestation: Agnès Dechartres approved the final manuscript.
Name: France Mentré, MD, PhD.
Contribution: This author helped design the study and write the manuscript.
Attestation: France Mentré has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
Name: Philippe Montravers, MD, PhD.
Contribution: This author helped write the manuscript.
Attestation: Philippe Montravers approved the final manuscript.
Name: Dan Longrois, MD, PhD.
Contribution: This author helped with the design of the manuscript and writing of the manuscript.
Attestation: Dan Longrois approved the final manuscript.
Name: Cedric Laouénan, MD.
Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.
Attestation: Cedric Laouénan has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.
This manuscript was handled by: Franklin Dexter, MD, PhD.
1. Grocott MP, Pearse RM. Perioperative medicine: the future of anaesthesia? Br J Anaesth. 2012;108:723–6
2. Poeran J, Mazumdar M, Memtsoudis SG. Anesthesia, outcomes, and public health: changing health care while “asleep.” Reg Anesth Pain Med. 2014;39:192–4
3. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RDPROGRESS Group. PROGRESS Group. . Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:e5595
4. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG, Hemingway HPROGRESS Group. PROGRESS Group. . Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10:e1001380
5. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DGPROGRESS Group. . Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10:e1001381
6. Hingorani AD, Windt DA, Riley RD, Abrams K, Moons KG, Steyerberg EW, Schroter S, Sauerbrei W, Altman DG, Hemingway HPROGRESS Group. PROGRESS Group. . Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ. 2013;346:e5793
7. Braitman LE, Davidoff F. Predicting clinical states in individual patients. Ann Intern Med. 1996;125:406–12
8. Katz MH. Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003;138:644–50
9. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87
10. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453–73
11. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605
12. Moons KG. Criteria for scientific evaluation of novel markers: a perspective. Clin Chem. 2010;56:537–41
13. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606
14. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338:b375
15. Steyerberg EW Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. 2010 New York, NY Springer
16. Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic BRadiation Therapy Oncology Group. Radiation Therapy Oncology Group. . Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ. 2004;328:22–4
17. Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable logistic models in the pulmonary and critical care literature. Chest. 2003;123:923–8
18. Mikolajczyk RT, DiSilvestro A, DiSilvesto A, Zhang J. Evaluation of logistic regression reporting in current obstetrics and gynecology literature. Obstet Gynecol. 2008;111:413–9
19. Mushkudiani NA, Hukkelhoven CW, Hernández AV, Murray GD, Choi SC, Maas AI, Steyerberg EW. A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes. J Clin Epidemiol. 2008;61:331–43
20. Kalil AC, Mattei J, Florescu DF, Sun J, Kalil RS. Recommendations for the assessment and reporting of multivariable logistic regression in transplantation literature. Am J Transplant. 2010;10:1686–94
21. Mallett S, Royston P, Dutton S, Waters R, Altman DG. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010;8:20
22. Mallett S, Royston P, Waters R, Dutton S, Altman DG. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21
23. Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103
24. Collins GS, Omar O, Shanyinde M, Yu LM. A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. J Clin Epidemiol. 2013;66:268–77
25. Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ. 2011;343:d7163
26. Bouwmeester W, Zuithoff NP, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KG. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9:1–12
27. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700
28. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ. 2009;338:b604
29. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JPSTROBE Initiative. STROBE Initiative. . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–7
30. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928–35
31. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54:17–23
32. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38
33. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130:515–24
34. Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Validity of prognostic models: when is a model clinically useful? Semin Urol Oncol. 2002;20:96–107
35. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058–64
37. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365
38. Vandenbroucke JP. STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK, and CONSORT: for whom does the guideline toll? J Clin Epidemiol. 2009;62:594–6
39. Cobo E, Cortés J, Ribera JM, Cardellach F, Selva-O’Callaghan A, Kostov B, García L, Cirugeda L, Altman DG, González JA, Sànchez JA, Miras F, Urrutia A, Fonollosa V, Rey-Joly C, Vilardell M. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;343:d6783
40. Altman DG, McShane LM, Sauerbrei W, Taube SE. Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): explanation and elaboration. PLoS Med. 2012;9:e1001216
41. Little R. Regression with missing X’s: a review. J Am Stat Assoc. 1992;87:1227–37
42. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–9
43. Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64:993–1000
44. Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ. 2006;332:1080
45. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25:127–41
46. Avram MJ, Shanks CA, Dykes MH, Ronai AK, Stiers WM. Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods. Anesth Analg. 1985;64:607–11
47. Nagele P. Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. Br J Anaesth. 2003;90:514–6
© 2015 International Anesthesia Research Society
48. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7