Share this article on:

Reporting and Methodology of Multivariable Analyses in Prognostic Observational Studies Published in 4 Anesthesiology Journals: A Methodological Descriptive Review

Guglielminotti, Jean MD, PhD*; Dechartres, Agnès MD, PhD; Mentré, France MD, PhD; Montravers, Philippe MD, PhD§; Longrois, Dan MD, PhD§; Laouénan, Cedric MD

doi: 10.1213/ANE.0000000000000517
Economics, Education, and Policy: Research Report

BACKGROUND: Prognostic research studies in anesthesiology aim to identify risk factors for an outcome (explanatory studies) or calculate the risk of this outcome on the basis of patients’ risk factors (predictive studies). Multivariable models express the relationship between predictors and an outcome and are used in both explanatory and predictive studies. Model development demands a strict methodology and a clear reporting to assess its reliability. In this methodological descriptive review, we critically assessed the reporting and methodology of multivariable analysis used in observational prognostic studies published in anesthesiology journals.

METHODS: A systematic search was conducted on Medline through Web of Knowledge, PubMed, and journal websites to identify observational prognostic studies with multivariable analysis published in Anesthesiology, Anesthesia & Analgesia, British Journal of Anaesthesia, and Anaesthesia in 2010 and 2011. Data were extracted by 2 independent readers. First, studies were analyzed with respect to reporting of outcomes, design, size, methods of analysis, model performance (discrimination and calibration), model validation, clinical usefulness, and STROBE (i.e., Strengthening the Reporting of Observational Studies in Epidemiology) checklist. A reporting rate was calculated on the basis of 21 items of the aforementioned points. Second, they were analyzed with respect to some predefined methodological points.

RESULTS: Eighty-six studies were included: 87.2% were explanatory and 80.2% investigated a postoperative event. The reporting was fairly good, with a median reporting rate of 79% (75% in explanatory studies and 100% in predictive studies). Six items had a reporting rate <36% (i.e., the 25th percentile), with some of them not identified in the STROBE checklist: blinded evaluation of the outcome (11.9%), reason for sample size (15.1%), handling of missing data (36.0%), assessment of colinearity (17.4%), assessment of interactions (13.9%), and calibration (34.9%). When reported, a few methodological shortcomings were observed, both in explanatory and predictive studies, such as an insufficient number of events of the outcome (44.6%), exclusion of cases with missing data (93.6%), or categorization of continuous variables (65.1%.).

CONCLUSIONS: The reporting of multivariable analysis was fairly good and could be further improved by checking reporting guidelines and EQUATOR Network website. Limiting the number of candidate variables, including cases with missing data, and not arbitrarily categorizing continuous variables should be encouraged.

Published ahead of print November 11, 2014

From the *Département d’Anesthésie-Réanimation Chirurgicale, APHP, Hôpital Bichat-Claude Bernard, Paris, France; INSERM, UMR 1137, IAME, Paris, France; Centre d’Epidemiologie Clinique, Hopital Hotel-Dieu, Paris, France; Université Paris Descartes, Sorbonne Paris Cité, Paris, France; Service de Biostatistique, APHP, Hôpital Bichat-Claude Bernard, Paris, France; Université Paris Diderot, Sorbonne Paris Cité, Paris, France; INSERM, UMR 1137, IAME, Paris, France; and §Département d’Anesthésie-Réanimation Chirurgicale, APHP, Hôpital Bichat-Claude Bernard, Paris, France; Université Paris Diderot, Sorbonne Paris Cité, Paris, France.

Accepted for publication August 27, 2014.

Published ahead of print November 11, 2014

Funding: Internally funded.

The authors declare no conflicts of interest.

Reprints will not be available from the authors.

Address correspondence to Jean Guglielminotti, MD, PhD, Département d’Anesthésie-Réanimation Chirurgicale, APHP, Hôpital Bichat-Claude Bernard, 46 Rue Henri Huchard, 75 877 Paris Cedex 18, France. Address e-mail to jean.guglielminotti@bch.aphp.fr.

The conceptual framework and practice of anesthesiology have changed during recent years. They have evolved from “anesthesia in the operating room” to “perioperative medicine,” which takes into consideration short- and long-term outcomes after anesthesia and surgery.1,2 The corollary is a growing concern in explaining and predicting patient outcomes. This is part of the rapidly expanding field of prognosis research in medicine.3–6 Prognostic factor research aims to identify risk factors for the outcome under study or to evaluate the association between 1 hypothesized risk factor of interest and the outcome.3,4 Prognostic model research aims to predict individual outcome on the basis of a patient’s risk factors to stratify the patient’s risk.3,5–7 Multivariable analysis expresses the relationship between the presence of clinical, biological, or genetic information (independent or explanatory variables or predictors) and a clinical state or outcome (dependent or explained variable).7 Multivariable analysis is therefore used in both prognostic factor and prognostic model research.8,9

For binary outcomes (i.e., the outcome did or did not occur), logistic regression and Cox (i.e., time-to-event) multivariable analyses are used. The development of a multivariable analysis for a binary outcome relies on a strict methodology to provide accurate identification of predictors or reliable risk estimates.10–15 In brief, development of a multivariable analysis follows successive steps: clear and explicit definition of the outcome, choice of the population, selection and coding of the explanatory variables, conduction of the statistical analysis, and assessment of model performance. For predictive studies, 2 additional steps are required: validation of the model and assessment of clinical usefulness.15 Moreover, clear and transparent reporting of the development and results of the analysis is mandatory to objectively judge its reliability and its potential usefulness for clinical practice.16 There is a growing concern that many multivariable analyses published in medical journals are poorly developed and reported.3–6,17–26 Reporting and methodology of multivariable analyses published in anesthesiology journals have never been assessed.

Promoting a better reporting and methodology of multivariable analyses in perioperative medicine could make their use more effective for the decision-making process. The aim of this methodological descriptive review was therefore to critically assess the reporting and methodology of multivariable analyses used in observational prognostic studies published in 4 leading anesthesiology journals.

Back to Top | Article Outline

METHODS

This methodological descriptive review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (i.e., PRISMA) guidelines.27 Registration of the review was deemed unnecessary by PROSPERO administrator since it was a descriptive review (http://www.crd.york.ac.uk/Prospero/).

Back to Top | Article Outline

Identification and Selection of Studies

We searched the Medline database through Web of Knowledge and PubMed from January 1, 2010, to December 31, 2011, for studies with multivariable analyses published in the 4 leading anesthesiology journals: Anesthesiology, British Journal of Anaesthesia, Anesthesia & Analgesia, and Anaesthesia. The search strings are given in Appendix 1. Moreover, an additional search was conducted on the website of the 4 journals with the key words reported in Appendix 1. These 4 journals were selected because they have the greatest 5-year impact factor among general anesthesiology journals. Web of Knowledge, PubMed, and Journal websites were last accessed on January 7, 2014, January 14, 2014, and January 21, 2014, respectively. No contact with study authors was made.

The retrieved articles were screened on the title and abstract by 1 of the investigators. Articles were eligible if they fulfilled the 4 following criteria: (1) original investigation including cohort, case-control, or registry studies and excluding nonobservational studies, case reports, guidelines, conference, meta-analyses, or systematic reviews; (2) human investigations; (3) topic related to anesthesiology or perioperative medicine; (4) and presence of a multivariable analysis in the study with a logistic regression model or a Cox model suggested by the presence in the title or the abstract of 1 of the following key words: odds ratio, hazard ratio, multivariable analysis, logistic regression, Cox model, prediction model, risk factors, and adjustment.

The eligible studies were independently assessed for inclusion on full text by 1 anesthesiologist and 1 biostatistician. After discussion and agreement between the 2 investigators, studies were excluded in the 3 following cases: (1) the study was unrelated to anesthesiology or perioperative medicine; (2) there was no multivariable analysis with a logistic regression model or a Cox model in the study; (3) the multivariable analysis was a secondary analysis (i.e., the multivariable analysis was an additional statistical tool to confirm a result obtained with another statistical method).

Back to Top | Article Outline

Data Collection Process and Data Items

A data-extraction form was designed specifically for evaluation of multivariable analyses with logistic regression or Cox models through consensus among the investigators and following published recommendations.5,11,13–15,28 The form was pretested on a sample of articles not included in this review by 2 of the investigators. In articles that presented >1 model, only the main model was assessed. Data were independently extracted by 2 investigators and entered into a database after disagreements between the 2 investigators had been resolved by discussion. If no agreement could be reached, a third investigator decided.

The articles also were assessed with the 22-item Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement checklist, especially the 4 items addressing the reporting of multivariable analysis: item 7 (variables), item 10 (study size), item 11 (quantitative variables), and item 12 (statistical methods).29

Studies were classified into 3 categories: (1) explanatory studies without a predefined hypothesis that aim to identify risk factors for the outcome under study (“net fishing”); (2) explanatory studies with a predefined hypothesis that evaluated the association between a defined risk factor and the outcome; and (3) predictive studies that calculated the individual probability of the outcome based on patient risk factors in order to stratify the individual risk.

The form was designed to assess, first, the reporting of items related to the 7 points described to follow, and second, when reported, 4 predefined methodological points.

Back to Top | Article Outline

General Characteristics of the Studies

We recorded the location of the study, its duration, and its funding source.

Back to Top | Article Outline

Outcomes

We recorded the nature of the primary outcome, its onset (i.e., pre-, intra-, or postoperative), the number of events for this outcome, and whether a clear and explicit definition was provided. When the primary outcome was not death or survival, we recorded whether the assessment was blinded. We also recorded whether secondary outcomes were studied and the number of secondary outcomes.

Back to Top | Article Outline

Design and Size

We recorded the study design (i.e., retrospective cohort, prospective cohort…), whether justification for sample size taking into consideration the number of events or the ratio of number of events to number of candidates was provided, the number of eligible patients (i.e., patients meeting inclusion criteria), and the number of included patients. When information was available, we calculated the ratio of number of included to number of eligible patients. This ratio can be helpful in assessing the potential for selection bias.

Back to Top | Article Outline

Multivariable Analysis

We recorded how candidate variables were selected (i.e., based on literature or investigator choice, based on statistical analysis, or based on both), the number of candidate variables tested, whether an explicit coding for all candidate variables was provided, how continuous candidate variables were coded, and whether colinearity among candidate variables was tested. When information was available, we calculated the ratio of number of events to number of candidate variables.

We recorded the method used to select variables in multivariable analysis (i.e., full model, backward selection, stepwise selection, or forward selection), whether interactions among variables were tested, and whether the authors reported a method for handling missing data. For missing data, complete case analysis refers to the fact that individuals with missing data regarding 1 or more candidates are excluded.

We also recorded whether an explicit coding for all final variables was provided, and whether odds and hazards ratio with their 95% confidence intervals were reported.

Back to Top | Article Outline

Performance

We recorded whether discrimination and calibration were reported.30–32 Discrimination refers to the model’s ability to discriminate a patient with the outcome from a patient without the outcome. It is usually evaluated with the c-index and may take any value between 0.5 (no discrimination) and 1 (perfect discrimination). Calibration refers to the agreement between the probability of the outcome predicted by the model and the observed probability in the studied population. An example of a commonly used calibration test is the Hosmer-Lemeshow goodness-of-fit test.

Back to Top | Article Outline

Validation

We recorded whether validation of the model was reported. Validation is a process of establishing that the model works satisfactorily for patients other than the original dataset used to develop the model.10,11,33 In order of increased levels of evidence, validation includes (1) internal validation using a split or resampling (bootstrapping); (2) external temporal validation using a completely different cohort of individuals than those used to derive the model but from the same institution; and (3) external geographical validation using a completely different cohort of individuals from a different institution.10,11,33 Although validation is particularly relevant for prediction models, internal validation can sometimes be used for explanatory models.

Back to Top | Article Outline

Clinical Usefulness

Assessment of clinical usefulness was considered present if decision analytic techniques such as the error rate or decision curve analysis were performed.15,32,34

Back to Top | Article Outline

Methodological Items

When reported, the 4 following methodological points were recorded: selection of candidate variables based on statistical screening, ratio of number of events to number of candidate variables <10, exclusion of cases with missing data (complete case analysis), and categorization of continuous candidate variables.

Back to Top | Article Outline

Assessment of the Risk of Bias in Individual Studies

Because no items currently are recommended to assess the risk of bias for multivariable analysis in individual studies, we used the Cochrane Collaboration’s tool for assessing risk of bias (http://www.cochrane-handbook.org) to determine items that could be considered as possible sources of bias in each study. We defined the following bias:

  1. Selection bias: no clear reporting of the ratio of number of included patients to number of eligible patients (i.e., number of patients with the inclusion criteria of the study).
  2. Detection bias (blinding of outcome assessment): no blinding of the method of assessment of the primary outcome except when the primary outcome was death.
  3. Reporting bias (selective reporting): no clear reporting of coding of candidate variables, no clear reporting of coding of final variables, no clear reporting of final odds or hazard ratio with their 95% confidence interval, and >1 outcome studied. For explanatory studies with an a priori hypothesis and negative results, we analyzed whether the authors in the abstract, results, discussion, or conclusion sections focused on statistically significant results not related to the hypothesis tested. This can be considered a form of spin initially described in randomized trials.35 Spin can be defined as specific reporting that could distort the interpretation of results and mislead readers.
Back to Top | Article Outline

Statistics

Statistical analysis was made with the R software, version 2.12.0 (R Foundation for Statistical Computing, Vienna, Austria). Descriptive statistics are presented as number (%) or median (27th−75th percentiles). A reporting rate common to predictive and explanatory studies was calculated. It included 21 items of reporting common to both types of studies.

Back to Top | Article Outline

RESULTS

Study Selection and General Characteristics

The flow diagram of the studies is presented in Figure 1.

Figure 1

Figure 1

Table 1

Table 1

Among the 86 studies included in this review, 52.9% were from North America, 82.4% were academia driven, and 80.2% investigated a postoperative event (Table 1). They were mainly explanatory with a tested hypothesis (68.6%) and based on logistic regression (80.2%). Bibliographic references of the studies included are presented in Appendix 2, and their individual description in Appendices 3–5. Slight but not striking differences were observed among the 4 journals included in this review (Appendix 6).

Back to Top | Article Outline

Reporting and STROBE Checklist

Among the 21 items of reporting common to both predictive and explanatory studies, the median reporting rate was 79% (27th−75th percentiles, 36–91) (Table 2). It was 100% (82–100) in predictive studies, 75% (44–87) in explanatory studies without tested hypothesis, and 76% (25–90) in explanatory studies with tested hypothesis.

Table 2

Table 2

Six items had a reporting rate <36% (i.e., the 25th percentile), with some of them not identified with the STROBE checklist: evaluation of the outcome (11.9%), reason for sample size (15.1%), handling of missing data (36.0%), assessment of colinearity (17.4%), interactions (13.9%), and calibration (34.9%).

The results of the assessment of the included articles with the STROBE checklist are presented in Appendix 7. The reporting of the description of all statistical methods was considered present in 95.3% of the studies.

Back to Top | Article Outline

Assessment of the Risk of Bias in Individual Studies

Table 3

Table 3

The individual risk of bias was small except for the lack of blinded assessment of the primary outcome (88.1%), and to a lesser extent, for >1 outcome studied (46.5%) and the presence of spin (42.1%) (Table 3).

Back to Top | Article Outline

Methodology

Table 4

Table 4

The selection of candidate variables for inclusion in the multivariable analysis based on statistical screening only was observed in 27.5% of the studies (Table 4), the ratio of the number of events to the number of candidate variables <10 in 44.6%, complete case analysis of missing data in 93.6%, and categorization of 1 or all continuous candidates in 65.1%.

Back to Top | Article Outline

DISCUSSION

This methodological descriptive review identified an acceptably good reporting of multivariable analysis with a median reporting rate of 79%. Some bias and methodological shortcomings, especially observed in explanatory studies, may hinder the reliability of the analysis but could be easily corrected.

Back to Top | Article Outline

Reporting

Contrasting with this good reporting rate, some points were obviously underreported, such as evaluation of the outcome, reason for sample size, handling of missing data, and assessment of colinearity, interactions, and performance. Complete and accurate reporting of the multivariable analysis should avoid the bias of assessing the quality of studies based only on the quality of their reporting.16 Moreover, it should also allow the reader to assess the applicability of the study to his/her own clinical practice. Poor reporting may lead to inappropriate decisions and consequences.36 There is evidence in effectiveness research that demonstrates the consequences of bad reporting.37 The studies included in this review were too recent to have led to inappropriate recommendations based on bad reporting.

The ever-growing plethora of available reporting guidelines may confuse and discourage the authors (Consolidated Standards of Reporting Trials [CONSORT] for randomized controlled trials, STROBE for observational studies…).38 The EQUATOR network website provides a comprehensive and updated list of statements and checklists for reporting adapted to each study design (http://www.equator-network.org/). This website should be consulted by authors when they report the results of their study. It may also help young investigators design their own studies. However, authors are probably not the only ones to be accountable for underreporting. Reviewers, editors, and publishers share responsibility and must contribute to the improvement of the current reporting. For instance, Cobo et al.39 demonstrated that using reporting guidelines during the peer-review process increased the quality of final manuscripts submitted to a biomedical journal.

The results of the analysis of the studies with the STROBE checklist focusing on multivariable analysis differ on some items from the results of the analysis with our assessment form. With the STROBE checklist, the reporting of the description of all statistical methods was considered present in 95.3% of the included studies. With our analysis, some aspects of the multivariable analysis were clearly underreported. To date, Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK) recommendations for the reporting of tumor marker prognostic studies and PROGnosis RESearch Strategy (PROGRESS) recommendations for prognosis research strategy are the most comprehensive and appropriate guidelines for the reporting of multivariable analyses.3–6,40 Reminders of recommendations for reporting of multivariable analysis based on STROBE, REMARK, and PROGRESS recommendations are presented in Table 5.

Table 5

Table 5

Some aspects of reporting recommended by these statements may, however, not be applicable to explanatory studies. For instance, one does not expect to see an internal or external validation in an explanatory study. However, the items of reporting, up to the stage of validation and clinical usefulness, are common to both types of studies. The reminder in Table 5 clearly differentiates the level of reporting for an explanatory and a predictive study. In addition, for explanatory studies with tested hypothesis, the only odds or hazard ratio of direct relevance is the one that assesses the hypothesized association between the risk factor and the outcome, adjusted for the other variables. Reporting and discussing odds or hazard ratio for all the adjusting variables used in this type of study may confuse the reader and increase the probability of spin.

Back to Top | Article Outline

Assessment of Bias in Individual Studies

The individual risk of bias was high for the blinded assessment of the primary outcome (detection bias), and, to a lesser extent, for the number of outcomes studied (reporting bias) and the presence of spin. If measurement of the outcome involves observer interpretation, it should be measured without knowledge of the candidate variables to avoid detection bias, especially in prospective studies. However, when the outcome is all-cause mortality (but not cause-specific mortality), blinding of the assessor is of lesser concern. In the current review, the measurement of the primary outcome was not reported as blinded in 88.1% of the studies in which the primary outcome was not death. When many outcomes are examined, especially when outcomes are not a priori defined, there is potential for bias in the selection of outcomes for multivariable analysis. In this review, 46.5% of the studies reported >1 outcome. Study registration and publication of the primary and secondary outcome measures and of the planned statistical analysis plan could prevent selective reporting and publication bias. Similar to controlled trials with statistically nonsignificant results for primary outcomes, 42.1% of explanatory studies with an a priori hypothesis and negative results for the hypothesis had spin.35

Back to Top | Article Outline

Methodology

There is no consensus on the best method for selecting candidate explanatory variables entered in the multivariable analysis. Some methods are particularly cautioned against, such as inclusion or exclusion of variables based only on univariable analysis observed in 27.5% of the included studies. It can be important to retain candidate variables known to be important from the literature but that may not reach statistical significance in a particular dataset. It is recommended that candidate variables that have clinical relevance or are already well established in the literature should be retained in models.15 Another concern with candidate variables is colinearity, which refers to the fact that variables may be strongly correlated with each other.15 Colinearity hampers reliable estimation of regression coefficients of the correlated variables. In the current review, only 17.4% of the studies reported that colinearity among candidate variables was assessed. The mathematical formulation of logistic regression or Cox models implies an additive effect of the variables on the outcome considered. However, the effect of a variable may depend on the effect of another one, a phenomenon called a 2-way interaction. In this case, the effect of 1 variable cannot be interpreted alone. In the current review, only 14% of the studies reported that interactions among variables were assessed.

Collecting all data on explanatory variables and outcomes for all patients rarely is achieved. A common approach to handle missing data is to exclude individuals with missing values and conduct a complete case analysis. However, this approach discards useful information and can lead to biased results.41 Imputation techniques are a valid approach to minimize the effect of missing data.15 In this review, information regarding missing data was reported in only 36% of the studies. Only 2 studies used imputation techniques. To assess the representativeness and quality of the data, reporting the number of missing data by variable and the number of individuals with complete data on all variables is advised.40

The predictive accuracy of a prediction model is determined by the number of events of the outcome (“effective sample size”) and not by the number of patients included in the study (“real sample size”).15 Although models in exploratory studies often have a more liberal approach than in predictive ones (“black box approach” versus “parsimonious approach” and restriction of predictors), the general recommendation is to have at least 10 events or even 20 of the studied outcome per explanatory variables. In this review, only 55.4% had a ratio >10 and 38.5% had >20. This “rule of thumb” should be considered when defining the study sample size.42,43 Ignoring this rule, especially in small sample size studies, leads to biased regression coefficients. In this review, in only 15.1% of the studies did authors report a justification for sample size based on the number of events or on the ratio of number of events to number of candidates.

The final model can be strongly affected by the coding of the variables, especially continuous variables.8,9,28 The practice of dichotomizing continuous candidate explanatory variables at the median value or at an arbitrary cutoff is not recommended, because it causes loss of information and statistical power. In addition, it results in unrealistic steps in the predicted risk, with patients at either side of a cutoff point categorized with very different levels of risk.44,45 In this review, categorization of a continuous variable was used in 65.1% of the studies. Continuous candidate variables should be kept continuous in the model. If the variable has a nonlinear relationship with the outcome, use of splines or fractional polynomial functions is recommended.40

Back to Top | Article Outline

Limits of the Study

The arbitrary choice to include the 4 general anesthesiology journals with the greatest impact factor may have biased the results. To the best of our knowledge, no study has specifically examined the relationship between impact factor and the quality of statistical methods in anesthesiology journals. Two studies indirectly indicate that the quality of statistical methods does not vary among anesthesiology journals, and we did not observe striking differences among the journals included in this review46,47 (Appendix 6).

Limiting this review to a 2-year period did not allow the assessment of temporal trends. However, the assessment of reporting and methodology of multivariable analysis in anesthesiology journals was never done before. This review should therefore be viewed as a first step in this assessment and as an incentive to redo it in a few years to determine whether the quality of reporting and methodology increases or decreases over time.

We deliberately chose not to contact authors because the aim of this study was not to assess the discrepancy between the reporting of the methodological aspects of the studies and what was really done by the authors.16 In other words, we wanted to assess what constitutes the available information for the common readers who have to analyze the study and make their own judgments. We considered that the reader of a study published in high impact factor journals such as the 4 journals included in this review should obtain the information required to assess the quality of the study without any additional effort. This point is all the more relevant because it is more and more difficult to keep up with the increasing amount of available information.48

We arbitrarily limited the method to consider confounding variables to statistical adjustment with multivariable analysis, because it is 1 of the most often used methods in medical literature. Other methods such as stratification, matching, or propensity scores can be used for this purpose, especially in explanatory studies.

Back to Top | Article Outline

CONCLUSIONS

The reporting of multivariable analysis in explanatory and predictive studies was acceptably good. Some points could be easily improved by the authors by checking reporting guidelines and the EQUATOR website and by editors by adopting editorial policies. Similarly, some methodological shortcomings could be easily corrected with limiting the number of candidate variables, including cases with missing data and not arbitrarily categorizing continuous variables.

Back to Top | Article Outline

APPENDIX 1

Search Strings for the Identification of Studies

  1. On Medline database through Web of Knowledge
  2. ((((TS=statistical models OR TS=risk factors OR TS=propensity score OR TS=multivariable analysis OR TS=decision making OR TS=validation studies OR TS=prognosis OR TS=outcome OR TS=prediction model OR TS=logistic regression OR TS=cox model) AND (TS=anesthesia OR TS=anesthesiology OR TS=perioperative care)))) AND Language=(English)
  3. Refined by: Source Titles=(ANESTHESIA AND ANALGESIA OR ANESTHESIOLOGY OR BRITISH JOURNAL OF ANAESTHESIA OR ANAESTHESIA)
  4. Timespan=2010–2011
  5. Databases=MEDLINE
  6. Lemmatization=On
  7. In this syntax (e.g., TS=risk factors), risk factors (without quoting) corresponds to “risk factors” and not to “risk or factors or risk factors.” TS is short for topic field. TS=risk factors find records containing the term risk factors in the Abstract, Title, or Key words. This is not a full-text search.
  8. On PubMed
  9. (“statistical models” OR “ risk factors” OR “propensity score” OR “multivariable analysis” OR “decision making” OR “validation studies” OR “prognosis” OR “outcome” OR “prediction model” OR “logistic regression” OR “cox model”)
  10. AND (“anesthesia” OR “anesthesiology” OR “perioperative care”)
  11. AND (“Anesthesiology”[Journal] OR “british journal of anaesthesia”[Journal] OR “Anesthesia and analgesia”[Journal] OR “Anaesthesia”[Journal])
  12. AND (“2010/01/01”[PDAT]: “2011/12/31”[PDAT])
  13. On the websites of the anesthesia journals

Statistical models OR risk factors OR propensity score OR multivariate analysis OR decision making OR validation studies OR prognosis OR outcome OR prediction model OR logistic regression OR cox model

Back to Top | Article Outline

APPENDIX 2

Bibliographic References of the 86 Studies Included in the Review

1. Aasvang EK, Gmaehle E, Hansen JB, Gmaehle B, Forman JL, Schwarz J, Bittner R, Kehlet H. Predictive risk factors for persistent postherniotomy pain. Anesthesiology 2010;112:957–69

2. Amar D, Munoz D, Shi W, Zhang H, Thaler HT. A clinical prediction rule for pulmonary complications after thoracic surgery for primary lung cancer. Anesth Analg 2010;110:1343–8

3. Argalious M, Xu M, Sun Z, Smedira N, Koch CG. Preoperative statin therapy is not associated with a reduced incidence of postoperative acute kidney injury after cardiac surgery. Anesth Analg 2010;111:324–30

4. Aronson S, Dyke CM, Levy JH, Cheung AT, Lumb PD, Avery EG, Hu MY, Newman MF. Does perioperative systolic blood pressure variability predict mortality after cardiac surgery? An exploratory analysis of the ECLIPSE trials. Anesth Analg 2011;113:19–30

5. Aronson S, Stafford-Smith M, Phillips-Bute B, Shaw A, Gaca J, Newman M; Cardiothoracic Anesthesiology Research Endeavors. Intraoperative systolic blood pressure variability predicts 30-day mortality in aortocoronary bypass surgery patients. Anesthesiology 2010;113:305–12

6. Ausset S, Auroy Y, Verret C, Benhamou D, Vest P, Cirodde A, Lenoir B. Quality of postoperative care after major orthopedic surgery is correlated with both long-term cardiovascular outcome and troponin Ic elevation. Anesthesiology 2010;113:529–40

7. Aziz MF, Healy D, Kheterpal S, Fu RF, Dillman D, Brambrink AM. Routine clinical practice effectiveness of the Glidescope in difficult airway management: an analysis of 2,004 Glidescope intubations, complications, and failures from two institutions. Anesthesiology 2011;114:34–41

8. Bateman BT, Berman MF, Riley LE, Leffert LR. The epidemiology of postpartum hemorrhage in a large, nationwide sample of deliveries. Anesth Analg 2010;110:1368–73

9. Benzon HT, Avram MJ, Benzon HA, Kirby-Nolan M, Nader A. Factor VII levels and international normalized ratios in the early phase of warfarin therapy. Anesthesiology 2010;112:298–304

10. Besch G, Liu N, Samain E, Pericard C, Boichut N, Mercier M, Chazot T, Pili-Floury S. Occurrence of and risk factors for electroencephalogram burst suppression during propofol-remifentanil anaesthesia. Br J Anaesth 2011;107:749–56

11. Boutonnet M, Faitot V, Katz A, Salomon L, Keita H. Mallampati class changes during pregnancy, labour, and after delivery: can these be predicted? Br J Anaesth 2010;104:67–70

12. Brandom BW, Larach MG, Chen MS, Young MC. Complications associated with the administration of dantrolene 1987 to 2006: a report from the North American Malignant Hyperthermia Registry of the Malignant Hyperthermia Association of the United States. Anesth Analg 2011;112:1115–23

13. Canet J, Gallart L, Gomar C, Paluzie G, Vallès J, Castillo J, Sabaté S, Mazo V, Briones Z, Sanchis J; ARISCAT Group. Prediction of postoperative pulmonary complications in a population-based surgical cohort. Anesthesiology 2010;113:1338–50

14. Chang CC, Lin HC, Lin HW, Lin HC. Anesthetic management and surgical site infections in total hip or knee replacement: a population-based study. Anesthesiology 2010;113:279–84

15. Collyer TC, Reynolds HC, Truyens E, Kilshaw L, Corcoran T. Perioperative management of clopidogrel therapy: the effects on in-hospital cardiac morbidity in older patients with hip fractures. Br J Anaesth 2011;107:911–5

16. Dalton JE, Kurz A, Turan A, Mascha EJ, Sessler DI, Saager L. Development and validation of a risk quantification index for 30-day postoperative mortality and morbidity in noncardiac surgical patients. Anesthesiology 2011;114:1336–44

17. Davidson AJ, Smith KR, Blussé van Oud-Alblas HJ, Lopez U, Malviya S, Bannister CF, Galinkin JL, Habre W, Ironfield C, Voepel-Lewis T, Weber F. Awareness in children: a secondary analysis of five cohort studies. Anaesthesia 2011;66:446–54

18. Deiner SG, Kwatra SG, Lin HM, Weisz DJ. Patient characteristics and anesthetic technique are additive but not synergistic predictors of successful motor evoked potential monitoring. Anesth Analg 2010;111:421–5

19. DiMaggio C, Sun LS, Li G. Early childhood exposure to anesthesia and risk of developmental and behavioral disorders in a sibling birth cohort. Anesth Analg 2011;113:1143–51

20. Duncan AE, Abd-Elsayed A, Maheshwari A, Xu M, Soltesz E, Koch CG. Role of intraoperative and postoperative blood glucose concentrations in predicting outcomes after cardiac surgery. Anesthesiology 2010;112:860–71

21. Durga P, Sahu BP, Mantha S, Ramachandran G. Development and validation of predictors of respiratory insufficiency and mortality scores: simple bedside additive scores for prediction of ventilation and in-hospital mortality in acute cervical spine injury. Anesth Analg 2010;110:134–40

22. Fernandez R, Tubau I, Masip J, Muñoz L, Roig I, Artigas A. Low reticulocyte hemoglobin content is associated with a higher blood transfusion rate in critically ill patients: a cohort study. Anesthesiology 2010;112:1211–5

23. Flick RP, Lee K, Hofer RE, Beinborn CW, Hambel EM, Klein MK, Gunn PW, Wilder RT, Katusic SK, Schroeder DR, Warner DO, Sprung J. Neuraxial labor analgesia for vaginal delivery and its effects on childhood learning disabilities. Anesth Analg 2011;112:1424–31

24. Flu WJ, van Kuijk JP, Hoeks SE, Kuiper R, Schouten O, Goei D, Elhendy A, Verhagen HJ, Thomson IR, Bax JJ, Fleisher LA, Poldermans D. Prognostic implications of asymptomatic left ventricular dysfunction in patients undergoing vascular surgery. Anesthesiology 2010;112:1316–24

25. Forget P, Vandenhende J, Berliere M, Machiels JP, Nussbaum B, Legrand C, De Kock M. Do intraoperative analgesics influence breast cancer recurrence after mastectomy? A retrospective analysis. Anesth Analg 2010;110:1630–5

26. Fox AA, Muehlschlegel JD, Body SC, Shernan SK, Liu KY, Perry TE, Aranki SF, Cook EF, Marcantonio ER, Collard CD. Comparison of the utility of preoperative versus postoperative B-type natriuretic peptide for predicting hospital length of stay and mortality after primary coronary artery bypass grafting. Anesthesiology 2010;112:842–51

27. Garvin S, Muehlschlegel JD, Perry TE, Chen J, Liu KY, Fox AA, Collard CD, Aranki SF, Shernan SK, Body SC. Postoperative activity, but not preoperative activity, of antithrombin is associated with major adverse cardiac events after coronary artery bypass graft surgery. Anesth Analg 2010;111:862–9

28. Glance LG, Wissler R, Mukamel DB, Li Y, Diachun CA, Salloum R, Fleming FJ, Dick AW. Perioperative outcomes among patients with the modified metabolic syndrome who are undergoing noncardiac surgery. Anesthesiology 2010;113:859–72

29. Glance LG, Dick AW, Mukamel DB, Fleming FJ, Zollo RA, Wissler R, Salloum R, Meredith UW, Osler TM. Association between intraoperative blood transfusion and mortality and morbidity in patients undergoing noncardiac surgery. Anesthesiology 2011;114:283–92

30. Gottschalk A, Ford JG, Regelin CC, You J, Mascha EJ, Sessler DI, Durieux ME, Nemergut EC. Association between epidural analgesia and cancer recurrence after colorectal cancer surgery. Anesthesiology 2010;113:27–34

31. Gupta A, Björnsson A, Fredriksson M, Hallböök O, Eintrei C. Reduction in mortality after epidural anaesthesia and analgesia in patients undergoing rectal but not colonic cancer surgery: a retrospective analysis of data from 655 patients in central Sweden. Br J Anaesth 2011;107:164–70

32. Haller G, Courvoisier DS, Anderson H, Myles PS. Clinical factors associated with the non-utilization of an anaesthesia incident reporting system. Br J Anaesth 2011;107:171–9

33. Hansen TG, Pedersen JK, Henneberg SW, Pedersen DA, Murray JC, Morton NS, Christensen K. Academic performance in adolescence after inguinal hernia repair in infancy: a nationwide cohort study. Anesthesiology 2011;114:1076–85

34. Heringlake M, Garbers C, Käbler JH, Anderson I, Heinze H, Schön J, Berger KU, Dibbelt L, Sievers HH, Hanke T. Preoperative cerebral oxygen saturation and clinical outcomes in cardiac surgery. Anesthesiology 2011;114:58–69

35. Hindman BJ, Bayman EO, Pfisterer WK, Torner JC, Todd MM; IHAST Investigators. No association between intraoperative hypothermia or supplemental protective drug and neurologic outcomes in patients undergoing temporary clipping during cerebral aneurysm surgery: findings from the Intraoperative Hypothermia for Aneurysm Surgery Trial. Anesthesiology 2010;112:86–101

36. Holmberg TJ, Bowman SM, Warner KJ, Vavilala MS, Bulger EM, Copass MK, Sharar SR. The association between obesity and difficult prehospital tracheal intubation. Anesth Analg 2011;112:1132–8

37. Hughes CG, Weavind L, Banerjee A, Mercaldo ND, Schildcrout JS, Pandharipande PP. Intraoperative risk factors for acute respiratory distress syndrome in critically ill patients. Anesth Analg 2010;111:464–7

38. Ismail H, Ho KM, Narayan K, Kondalsamy-Chennakesavan S. Effect of neuraxial anaesthesia on tumour progression in cervical cancer patients treated with brachytherapy: a retrospective cohort study. Br J Anaesth 2010;105:145–9

39. Jacob AK, Mantilla CB, Sviggum HP, Schroeder DR, Pagnano MW, Hebl JR. Perioperative nerve injury after total hip arthroplasty: regional anesthesia risk during a 20-year cohort study. Anesthesiology 2011;115:1172–8

40. Jankowski CJ, Trenerry MR, Cook DJ, Buenvenida SL, Stevens SR, Schroeder DR, Warner DO. Cognitive and functional predictors and sequelae of postoperative delirium in elderly patients undergoing elective joint arthroplasty. Anesth Analg 2011;112:1186–93

41. Jun NH, Shim JK, Kim JC, Kwak YL. Prognostic value of a tissue Doppler-derived index of left ventricular filling pressure on composite morbidity after off-pump coronary artery bypass surgery. Br J Anaesth 2011;107:519–24

42. Kertai MD, Pal N, Palanca BJ, Lin N, Searleman SA, Zhang L, Burnside BA, Finkel KJ, Avidan MS; B-Unaware Study Group. Association of perioperative risk factors and cumulative duration of low bispectral index with intermediate-term mortality after cardiac surgery in the B-Unaware Trial. Anesthesiology 2010;112:1116–27

43. Kertai MD, Palanca BJ, Pal N, Burnside BA, Zhang L, Sadiq F, Finkel KJ, Avidan MS; B-Unaware Study Group. Bispectral index monitoring, duration of bispectral index below 45, patient risk factors, and intermediate-term mortality after noncardiac surgery in the B-Unaware Trial. Anesthesiology 2011;114:545–56

44. Kim WH, Ahn HJ, Lee CJ, Shin BS, Ko JS, Choi SJ, Ryu SA. Neck circumference to thyromental distance ratio: a new predictor of difficult intubation in obese patients. Br J Anaesth 2011;106:743–8

45. Kin N, Weismann C, Srivastava S, Chakravarti S, Bodian C, Hossain S, Krol M, Hollinger I, Nguyen K, Mittnacht AJ. Factors affecting the decision to defer endotracheal extubation after surgery for congenital heart disease: a prospective observational study. Anesth Analg 2011;113:329–35

46. Kor DJ, Warner DO, Alsara A, Fernández-Pérez ER, Malinchoc M, Kashyap R, Li G, Gajic O. Derivation and diagnostic accuracy of the surgical lung injury prediction model. Anesthesiology 2011;115:117–28

47. Kraemer FW, Stricker PA, Gurnaney HG, McClung H, Meador MR, Sussman E, Burgess BJ, Ciampa B, Mendelsohn J, Rehman MA, Watcha MF. Bradycardia during induction of anesthesia with sevoflurane in children with Down syndrome. Anesth Analg 2010;111:1259–63

48. Le Manach Y, Ibanez Esteves C, Bertrand M, Goarin JP, Fléron MH, Coriat P, Koskas F, Riou B, Landais P. Impact of preoperative statin therapy on adverse postoperative outcomes in patients undergoing vascular surgery. Anesthesiology 2011;114:98–104

49. Leslie K, Myles PS, Forbes A, Chan MT. The effect of bispectral index monitoring on long-term survival in the B-aware trial. Anesth Analg 2010;110:816–22

50. Leslie K, Myles PS, Chan MT, Forbes A, Paech MJ, Peyton P, Silbert BS, Williamson E. Nitrous oxide and long-term morbidity and mortality in the ENIGMA trial. Anesth Analg 2011;112:387–93

51. Lin L, Liu C, Tan H, Ouyang H, Zhang Y, Zeng W. Anaesthetic technique may affect prognosis for ovarian serous adenocarcinoma: a retrospective analysis. Br J Anaesth 2011;106:814–22

52. Lindholm ML, Granath F, Eriksson LI, Sandin R. Malignant disease within 5 years after surgery in relation to duration of sevoflurane anesthesia and time with bispectral index under 45. Anesth Analg 2011;113:778–83

53. Lobo SM, Rezende E, Knibel MF, Silva NB, Páramo JA, Nácul FE, Mendes CL, Assunção M, Costa RC, Grion CC, Pinto SF, Mello PM, Maia MO, Duarte PA, Gutierrez F, Silva JM Jr, Lopes MR, Cordeiro JA, Mellot C. Early determinants of death due to multiple organ failure after noncardiac surgery in high-risk patients. Anesth Analg 2011;112:877–83

54. Martin LD, Mhyre JM, Shanks AM, Tremper KK, Kheterpal S. 3,423 emergency tracheal intubations at a university hospital: airway outcomes and complications. Anesthesiology 2011;114:42–8

55. Mashour GA, Shanks AM, Kheterpal S. Perioperative stroke and associated mortality after noncardiac, nonneurologic surgery. Anesthesiology 2011;114:1289–96

56. Mauermann WJ, Nuttall GA, Cook DJ, Hanson AC, Schroeder DR, Oliver WC. Hemofiltration during cardiopulmonary bypass does not decrease the incidence of atrial fibrillation after cardiac surgery. Anesth Analg 2010;110:329–34

57. McDonagh DL, Mathew JP, White WD, Phillips-Bute B, Laskowitz DT, Podgoreanu MV, Newman MF; Neurologic Outcome Research Group. Cognitive function after major noncardiac surgery, apolipoprotein E4 genotype, and biomarkers of brain injury. Anesthesiology 2010;112:852–9

58. McKenny M, Ryan T, Tate H, Graham B, Young VK, Dowd N. Age of transfused blood is not associated with increased postoperative adverse outcome after cardiac surgery. Br J Anaesth 2011;106:643–9

59. Meier PM, Goobie SM, DiNardo JA, Proctor MR, Zurakowski D, Soriano SG. Endoscopic strip craniectomy in early infancy: the initial five years of anesthesia experience. Anesth Analg 2011;112:407–14

60. Memtsoudis SG, Ma Y, Chiu YL, Walz JM, Voswinckel R, Mazumdar M. Perioperative mortality in patients with pulmonary hypertension undergoing major joint replacement. Anesth Analg 2010;111:1110–6

61. Memtsoudis SG, Ma Y, Chiu YL, Poultsides L, Gonzalez Della Valle A, Mazumdar M. Bilateral total knee arthroplasty: risk factors for major morbidity and mortality. Anesth Analg 2011;113:784–90

62. Mhyre JM, Ramachandran SK, Kheterpal S, Morris M, Chan PS; American Heart Association National Registry for Cardiopulmonary Resuscitation Investigators. Delayed time to defibrillation after intraoperative and periprocedural cardiac arrest. Anesthesiology 2010;113:782–93

63. Mhyre JM, Bateman BT, Leffert LR. Influence of patient comorbidities on the risk of near-miss maternal morbidity or mortality. Anesthesiology 2011;115:963–72

64. Moganasundram S, Hunt BJ, Sykes K, Holton F, Parmar K, Durward A, Murdoch IA, Austin C, Anderson D, Tibby SM. The relationship among thromboelastography, hemostatic variables, and bleeding after cardiopulmonary bypass surgery in children. Anesth Analg 2010;110:995–1002

65. Muehlschlegel JD, Perry TE, Liu KY, Fox AA, Collard CD, Shernan SK, Body SC. Heart-type fatty acid binding protein is an independent predictor of death and ventricular dysfunction after coronary artery bypass graft surgery. Anesth Analg 2010;111:1101–9

66. Nafiu OO, Kheterpal S, Moulding R, Picton P, Tremper KK, Campbell DA Jr, Eliason JL, Stanley JC. The association of body mass index to postoperative outcomes in elderly vascular surgery patients: a reverse J-curve phenomenon. Anesth Analg 2011;112:23–9

67. Nikolov NM, Fontes ML, White WD, Aronson S, Bar-Yosef S, Gaca JG, Podgoreanu MV, Stafford-Smith M, Newman MF, Mathew JP. Pulse pressure and long-term survival after coronary artery bypass graft surgery. Anesth Analg 2010;110:335–40

68. Noordzij PG, Poldermans D, Schouten O, Bax JJ, Schreiner FA, Boersma E. Postoperative mortality in The Netherlands: a population-based analysis of surgery-specific risk in adults. Anesthesiology 2010;112:1105–15

69. Powell ES, Cook D, Pearce AC, Davies P, Bowler GM, Naidu B, Gao F; UKPOS Investigators. A prospective, multicentre, observational cohort study of analgesia and outcome after pneumonectomy. Br J Anaesth 2011;106:364–70

70. Ramachandran SK, Kheterpal S, Consens F, Shanks A, Doherty TM, Morris M, Tremper KK. Derivation and validation of a simple perioperative sleep apnea prediction score. Anesth Analg 2010;110:1007–15

71. Ramachandran SK, Nafiu OO, Ghaferi A, Tremper KK, Shanks A, Kheterpal S. Independent predictors and outcomes of unanticipated early postoperative tracheal intubation after nonemergent, noncardiac surgery. Anesthesiology 2011;115:44–53

72. Ruiz JR, Kee SS, Frenzel JC, Ensor JE, Selvan M, Riedel BJ, Apfel C. The effect of an anatomically classified procedure on antiemetic administration in the postanesthesia care unit. Anesth Analg 2010;110:403–9

73. Sabaté S, Mases A, Guilera N, Canet J, Castillo J, Orrego C, Sabaté A, Fita G, Parramón F, Paniagua P, Rodríguez A, Sabaté M; ANESCARDIOCAT Group. Incidence and predictors of major perioperative adverse cardiac and cerebrovascular events in non-cardiac surgery. Br J Anaesth 2011;107:879–90

74. Schwann NM, Hillel Z, Hoeft A, Barash P, Möhnle P, Miao Y, Mangano DT. Lack of effectiveness of the pulmonary artery catheter in cardiac surgery. Anesth Analg 2011;113:994–1002

75. Sessler DI, Kurz A, Saager L, Dalton JE. Operation timing and 30-day mortality after elective general surgery. Anesth Analg 2011;113:1423–8

76. Shi Y, Warner DO. Pediatric surgery and parental smoking behavior. Anesthesiology 2011;115:12–7

77. Song JG, Jeong SM, Shin WJ, Jun IG, Shin K, Huh IY, Kim YK, Hwang GS. Laboratory variables associated with low near-infrared cerebral oxygen saturation in icteric patients before liver transplantation surgery. Anesth Analg 2011;112:1347–52

78. Tsai PS, Hsu CS, Fan YC, Huang CJ. General anaesthesia is associated with increased risk of surgical site infection after Caesarean delivery compared with neuraxial anaesthesia: a population-based study. Br J Anaesth 2011;107:757–61

79. van Lier F, van der Geest PJ, Hoeks SE, van Gestel YR, Hol JW, Sin DD, Stolker RJ, Poldermans D. Epidural analgesia is associated with improved health outcomes of surgical patients with chronic obstructive pulmonary disease. Anesthesiology 2011;115:315–21

80. Wallace AW, Au S, Cason BA. Association of the pattern of use of perioperative β-blockade and postoperative mortality. Anesthesiology 2010;113:794–805

81. Wallace AW, Au S, Cason BA. Perioperative β-blockade: atenolol is associated with reduced mortality when compared to metoprolol. Anesthesiology 2011;114:824–36

82. Wang JF, Bian JJ, Wan XJ, Zhu KM, Sun ZZ, Lu AD. NFKB1-94ins/del polymorphism is not associated with lung injury after cardiopulmonary bypass. Anaesthesia 2010;65:158–62

83. Weingarten TN, Flores AS, McKenzie JA, Nguyen LT, Robinson WB, Kinney TM, Siems BT, Wenzel PJ, Sarr MG, Marienau MS, Schroeder DR, Olson EJ, Morgenthaler TI, Warner DO, Sprung J. Obstructive sleep apnoea and perioperative complications in bariatric patients. Br J Anaesth 2011;106:131–9

84. Williams TA, Ho KM, Dobb GJ, Finn JC, Knuiman M, Webb SA; Royal Perth Hospital ICU Data Linkage Group. Effect of length of stay in intensive care unit on hospital and long-term mortality of critically ill adult patients. Br J Anaesth 2010;104:459–64

85. Wuethrich PY, Hsu Schmitz SF, Kessler TM, Thalmann GN, Studer UE, Stueber F, Burkhard FC. Potential influence of the anesthetic technique used during open radical prostatectomy on prostate cancer-related outcome: a retrospective study. Anesthesiology 2010;113:570–6

86. Yoo HS, Nahm FS, Lee PB, Lee CJ. Early thoracic sympathetic block improves the treatment effect for upper extremity neuropathic pain. Anesth Analg 2011;113:605–9

Back to Top | Article Outline

Appendix 3. Individual Results: General Characteristics and Primary Outcome

Table

Table

Back to Top | Article Outline

Appendix 4. Individual Results: Model Development

Table

Table

Back to Top | Article Outline

Appendix 5. Individual Results: Model Performance and Spin

Table

Table

Back to Top | Article Outline

Appendix 6. Comparison Among 3 of the 4 Journals Included in the Review on Some Points of Reporting and Methodology

Table

Table

Back to Top | Article Outline

Appendix 7. Assessment of the Included Articles in the Review with the STROBE Checklist for 4 Items Addressing the Reporting of Multivariable Analysis

Table

Table

Back to Top | Article Outline

DISCLOSURES

Name: Jean Guglielminotti, MD, PhD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Jean Guglielminotti has seen the original study data, reviewed the analysis of the data, approved the final manuscript, and is the author responsible for archiving the study files.

Name: Agnès Dechartres, MD, PhD.

Contribution: This author helped write the manuscript.

Attestation: Agnès Dechartres approved the final manuscript.

Name: France Mentré, MD, PhD.

Contribution: This author helped design the study and write the manuscript.

Attestation: France Mentré has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

Name: Philippe Montravers, MD, PhD.

Contribution: This author helped write the manuscript.

Attestation: Philippe Montravers approved the final manuscript.

Name: Dan Longrois, MD, PhD.

Contribution: This author helped with the design of the manuscript and writing of the manuscript.

Attestation: Dan Longrois approved the final manuscript.

Name: Cedric Laouénan, MD.

Contribution: This author helped design the study, conduct the study, analyze the data, and write the manuscript.

Attestation: Cedric Laouénan has seen the original study data, reviewed the analysis of the data, and approved the final manuscript.

This manuscript was handled by: Franklin Dexter, MD, PhD.

Back to Top | Article Outline

REFERENCES

1. Grocott MP, Pearse RM. Perioperative medicine: the future of anaesthesia? Br J Anaesth. 2012;108:723–6
2. Poeran J, Mazumdar M, Memtsoudis SG. Anesthesia, outcomes, and public health: changing health care while “asleep.” Reg Anesth Pain Med. 2014;39:192–4
3. Hemingway H, Croft P, Perel P, Hayden JA, Abrams K, Timmis A, Briggs A, Udumyan R, Moons KG, Steyerberg EW, Roberts I, Schroter S, Altman DG, Riley RDPROGRESS Group. PROGRESS Group. . Prognosis research strategy (PROGRESS) 1: a framework for researching clinical outcomes. BMJ. 2013;346:e5595
4. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, Malats N, Briggs A, Schroter S, Altman DG, Hemingway HPROGRESS Group. PROGRESS Group. . Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med. 2013;10:e1001380
5. Steyerberg EW, Moons KG, van der Windt DA, Hayden JA, Perel P, Schroter S, Riley RD, Hemingway H, Altman DGPROGRESS Group. . Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 2013;10:e1001381
6. Hingorani AD, Windt DA, Riley RD, Abrams K, Moons KG, Steyerberg EW, Schroter S, Sauerbrei W, Altman DG, Hemingway HPROGRESS Group. PROGRESS Group. . Prognosis research strategy (PROGRESS) 4: stratified medicine research. BMJ. 2013;346:e5793
7. Braitman LE, Davidoff F. Predicting clinical states in individual patients. Ann Intern Med. 1996;125:406–12
8. Katz MH. Multivariable analysis: a primer for readers of medical research. Ann Intern Med. 2003;138:644–50
9. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–87
10. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19:453–73
11. Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ. 2009;338:b605
12. Moons KG. Criteria for scientific evaluation of novel markers: a perspective. Clin Chem. 2010;56:537–41
13. Moons KG, Altman DG, Vergouwe Y, Royston P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ. 2009;338:b606
14. Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ. 2009;338:b375
15. Steyerberg EW Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. 2010 New York, NY Springer
16. Soares HP, Daniels S, Kumar A, Clarke M, Scott C, Swann S, Djulbegovic BRadiation Therapy Oncology Group. Radiation Therapy Oncology Group. . Bad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group. BMJ. 2004;328:22–4
17. Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable logistic models in the pulmonary and critical care literature. Chest. 2003;123:923–8
18. Mikolajczyk RT, DiSilvestro A, DiSilvesto A, Zhang J. Evaluation of logistic regression reporting in current obstetrics and gynecology literature. Obstet Gynecol. 2008;111:413–9
    19. Mushkudiani NA, Hukkelhoven CW, Hernández AV, Murray GD, Choi SC, Maas AI, Steyerberg EW. A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes. J Clin Epidemiol. 2008;61:331–43
      20. Kalil AC, Mattei J, Florescu DF, Sun J, Kalil RS. Recommendations for the assessment and reporting of multivariable logistic regression in transplantation literature. Am J Transplant. 2010;10:1686–94
        21. Mallett S, Royston P, Dutton S, Waters R, Altman DG. Reporting methods in studies developing prognostic models in cancer: a review. BMC Med. 2010;8:20
          22. Mallett S, Royston P, Waters R, Dutton S, Altman DG. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21
            23. Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103
              24. Collins GS, Omar O, Shanyinde M, Yu LM. A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. J Clin Epidemiol. 2013;66:268–77
              25. Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ. 2011;343:d7163
                26. Bouwmeester W, Zuithoff NP, Mallett S, Geerlings MI, Vergouwe Y, Steyerberg EW, Altman DG, Moons KG. Reporting and methods in clinical prediction research: a systematic review. PLoS Med. 2012;9:1–12
                  27. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JP, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700
                  28. Royston P, Moons KG, Altman DG, Vergouwe Y. Prognosis and prognostic research: Developing a prognostic model. BMJ. 2009;338:b604
                  29. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JPSTROBE Initiative. STROBE Initiative. . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370:1453–7
                  30. Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007;115:928–35
                  31. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54:17–23
                  32. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–38
                  33. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability of prognostic information. Ann Intern Med. 1999;130:515–24
                  34. Vergouwe Y, Steyerberg EW, Eijkemans MJ, Habbema JD. Validity of prognostic models: when is a model clinically useful? Semin Urol Oncol. 2002;20:96–107
                  35. Boutron I, Dutton S, Ravaud P, Altman DG. Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010;303:2058–64
                  36. EQUATOR. ACT Now: Accuracy, Completeness, and Transparency in Health Research Reporting EQUATOR Annual Lecture. 2012 Fribourg, Germany Available at http://www.equator-network.org/2012/10/10/act-now-accuracy-completeness-and-transparency-in-health-research-reporting/. Accessed October 30, 2014
                  37. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ. 2010;340:c365
                  38. Vandenbroucke JP. STREGA, STROBE, STARD, SQUIRE, MOOSE, PRISMA, GNOSIS, TREND, ORION, COREQ, QUOROM, REMARK, and CONSORT: for whom does the guideline toll? J Clin Epidemiol. 2009;62:594–6
                  39. Cobo E, Cortés J, Ribera JM, Cardellach F, Selva-O’Callaghan A, Kostov B, García L, Cirugeda L, Altman DG, González JA, Sànchez JA, Miras F, Urrutia A, Fonollosa V, Rey-Joly C, Vilardell M. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ. 2011;343:d6783
                  40. Altman DG, McShane LM, Sauerbrei W, Taube SE. Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): explanation and elaboration. PLoS Med. 2012;9:e1001216
                  41. Little R. Regression with missing X’s: a review. J Am Stat Assoc. 1992;87:1227–37
                  42. Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR. A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol. 1996;49:1373–9
                  43. Courvoisier DS, Combescure C, Agoritsas T, Gayet-Ageron A, Perneger TV. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. J Clin Epidemiol. 2011;64:993–1000
                  44. Altman DG, Royston P. The cost of dichotomising continuous variables. BMJ. 2006;332:1080
                  45. Royston P, Altman DG, Sauerbrei W. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med. 2006;25:127–41
                  46. Avram MJ, Shanks CA, Dykes MH, Ronai AK, Stiers WM. Statistical methods in anesthesia articles: an evaluation of two American journals during two six-month periods. Anesth Analg. 1985;64:607–11
                  47. Nagele P. Misuse of standard error of the mean (SEM) when reporting variability of a sample. A critical evaluation of four anaesthesia journals. Br J Anaesth. 2003;90:514–6
                  48. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7
                  © 2015 International Anesthesia Research Society