Coronavirus disease 2019 (COVID-19) can result in severe illness requiring critical care in around 5% of those with confirmed infection (1). Reported mortality for patients critically ill with COVID-19 has varied widely (2–6); however, prognostic research has been focused on models for identifying people in the general population at risk of being admitted to hospital with COVID-19, detecting likely COVID-19 in patients with suspected infection, and predicting prognosis of hospitalized patients with COVID-19 (7). Patients critically ill with COVID-19 have not been the focus of many prognostic studies, despite their increased risk of death compared with patients outside of critical care. Most studies have been based on small samples and few have considered data from countries outside China.
Since the outbreak in the United Kingdom (UK) started, the Intensive Care National Audit & Research Centre has been collecting data from critical care units participating in the Case Mix Programme (the national clinical audit covering all NHS adult, general intensive care, and combined intensive care/high dependency units in England, Wales, and Northern Ireland, plus some additional specialist and non-NHS critical care units) to support the NHS and policy makers with rapid data collection and analysis. The number of patients included and completeness of the population coverage make the Case Mix Programme an ideal resource to investigate factors associated with mortality in critical care. At the peak of the epidemic, 400 patients per day were being admitted to critical care with COVID-19 (8), exceeding the typical number of daily unplanned admissions to adult general critical care in the UK for all conditions combined.
The aim of the present study was to identify patient characteristics that predict death in hospital within 30 days of the start of critical care for patients critically ill with confirmed COVID-19 in England, Wales, and Northern Ireland.
MATERIALS AND METHODS
Selection of Data
We included all patients admitted for critical care between March 1, 2020, and June 22, 2020 with a diagnosis of COVID-19 confirmed either at or after the start of critical care. As physiologic and laboratory variables for the Case Mix Programme are recorded as the most extreme measurements within the first 24 hours following the start of critical care, patients with a duration of critical care of less than 24 hours were excluded. To avoid inclusion of patients with only minimal data for modeling, patients with no data recorded for age or sex or for the core physiologic variables of temperature, systolic blood pressure, heart rate (HR), or respiratory rate were also excluded from the model.
Support for the collection and use of patient-identifiable data without consent in the Case Mix Programme has been obtained from the Confidentiality Advisory Group of the Health Research Authority under Section 251 of the NHS Act 2006 (approval number PIAG 2–10[f]/2005).
Outcome and Prognostic Factors
The primary outcome was time to death within 30 days from the start of critical care. For those discharged from hospital within 30 days or ending critical care within 30 days with missing hospital outcome, we assumed all survived to 30 days. Patients last reported to be still receiving critical care were censored on July 22, 2020.
Candidate prognostic factors (eTable 1, Supplemental Digital Content 1, http://links.lww.com/CCM/F962) were selected a priori based on the established relationships with outcome for critically ill patients, emerging information from the COVID-19 pandemic, and availability within the Case Mix Programme dataset. Physiologic and laboratory variables were assessed as the most extreme values within the first 24 hours of critical care. Where patients were transferred between the critical care units (in the same or another hospital) or readmitted to critical care, physiologic, laboratory, and treatment variables were included from the first 24 hours of the first admission only.
Handling of Missing Data
Data were assumed to be missing at random conditional on the observed variables; this assumption was based on the premise that the decision to request additional tests would be based on the known information about that patient. Ten imputed datasets were produced using fully conditional specification (9). All candidate prognostic factors (with or without missing values) were included in the imputation model along with the failure indicator and the Nelson-Aalen estimate of the baseline cumulative hazard (10). Height and weight were imputed separately with body mass index (BMI) imputed passively. The model was fitted to each of the ten imputed datasets and the results were combined using Rubin rules (11). A sensitivity analysis was performed using the complete case dataset to examine the consistency of results.
We estimated a required sample size of 21 events per parameter (eTable 2, Supplemental Digital Content 2, http://links.lww.com/CCM/F963) (12). To allow for flexible modeling of nonlinear associations, we assumed that each continuous variable was represented by three parameters. Therefore, a model including 45 parameters would require a minimum of 945 events to minimize model overfitting and target sufficiently precise model predictions.
Association between the selected factors and time to death within 30 days of the start of critical care was determined after adjustment for multiple variables with Cox proportional hazards modeling. When departures from linearity were significant, we chose functional forms for continuous variables that gave the best plausible fit from restricted cubic splines with three, four, or five knots (13). No univariable preselection of variables was done and all candidates were introduced into the model. Expanded terms were used to test the interactions between mechanical ventilation and Pao2/Fio2, respiratory rate, HR, and creatinine. Interactions between mechanical ventilation and BMI and between ethnicity and hemoglobin were also assessed. We used a p value of 0.01 to retain interaction terms in the model. The results of the Cox proportional hazards model were plotted graphically for each prognostic factor. The reference value for continuous prognostic factors was chosen to represent the extreme of the normal range.
The proportional hazards assumption was tested by visual inspection of Schoenfeld residual plots (14) and log-log plots. Harrell concordance statistic (c-statistic) was used as a goodness-of-fit measure (15).
All statistical analyses were undertaken using Stata/SE 14.2 (StataCorp, College Station, TX).
Data were received for 12,773 admissions for 10,362 patients to 258 adult critical care units in England, Wales, and Northern Ireland between March 1, 2020, and June 22, 2020. We excluded 95 patients with a duration of critical care of less than 24 hours, 17 patients missing age or sex, and 260 patients with no data recorded for core physiologic variables resulting in a cohort of 9,990 patients for analysis (Fig. 1).
For the remaining physiologic and laboratory variables, the percentage with missing values ranged from 2.8% for the lowest hemoglobin concentration to 4.8% for the highest blood lactate concentration (eTable 3, Supplemental Digital Content 3, http://links.lww.com/CCM/F964). Ethnicity was either not stated or not recorded for 3.8% of patients and postcodes required to derive deprivation were not recorded for 7.2%. Height and weight for the calculation of BMI were missing for 5.2% and 3.2%, respectively. Overall, 7,670 patients (76.8%) had complete data for all variables.
The characteristics of included patients are summarized in Table 1. The median age was 60 years and 70% were male. Over one-third were of non-White ethnicities (compared with 13% of the UK population at the last national census in 2011) and half were from the areas with higher levels of deprivation (fourth or fifth quintile).
TABLE 1. -
Characteristics and Outcomes of Included Patients
|Characteristics and Outcomes
||All Patients (n = 9,990)
||Survivors at 30 d (n = 6,027)
||Nonsurvivors at 30 d (n =3,933)
| Age (yr), median (IQR)
| Sex, n (%)
| Ethnicity, n (%)
| Quintile of deprivation, n (%)
| 1 (least deprived)
| 5 (most deprived)
| Body mass index (kg/m2), median (IQR)
| Dependency prior to hospital admission, n (%)
| Able to live without assistance in daily activities
| Some (minor/major) assistance with daily activities
| Total assistance with all daily activities
| Immunocompromise, n (%)
|Acute severity scores from first 24 h of critical care
| Acute Physiology and Chronic Health Evaluation II score, median (IQR)
| Intensive Care National Audit & Research Centre physiology score, median (IQR)
|Interventions during first 24 hr of critical care, n (%)
| Sedated for entire of first 24 hr
| Mechanical ventilation at any time in first 24 hr
|Physiologic and laboratory variables from first 24 hr of critical care, median (IQR)
| Highest temperature (°C)
| Lowest systolic blood pressure (mm Hg)
| Highest heart rate (beats/min)
| Highest respiratory rate (breaths/min)
2 (mm Hg)
| Highest blood lactate concentration (mmol/L)
| Highest serum creatinine (µmol/L)
| Highest serum urea (mmol/L)
| Lowest hemoglobin concentration (g/L)
| Lowest platelet count (×109/L)
| Last known status at end of study, n (%)
| Died while receiving critical care
| Died in acute hospital following discharge from critical care
| Still receiving critical care
| Still in acute hospital
| Discharged alive from acute hospital
IQR = interquartile range, NA = not available.
At the end of the study period, 3,867 patients died while receiving critical care (3,679 within 30 d). A further 5,934 patients had been discharged alive from critical care and 290 of these patients subsequently died in hospital (254 within 30 d). The remaining 189 patients were last reported to be still receiving critical care at the end of the study, including 30 patients lost to follow-up following transfer to a different critical care unit. The 30-day outcome was, therefore, complete for 9,960 patients (99.7%). A Kaplan-Meier plot of time to death within 30 days is shown in eFigure 1 (Supplemental Digital Content 4, http://links.lww.com/CCM/F965).
Associations With 30-Day Mortality
The associations between the patient characteristics and 30-day mortality are presented graphically in eFigure 2 (Supplemental Digital Content 5, http://links.lww.com/CCM/F966), with full model output in Table 2.
TABLE 2. -
Multivariable Cox Proportional Hazards Model for Death Within 30 Days of Start of Critical Care (n
||Hazard Ratio (95% CI)
|Age (yr)—RCS (42, 60, 75)
| Spline base variable 1
| Spline base variable 2
|Male sex (vs female)
|Ethnicity (vs White)
|Quintile of deprivation (vs 1, least deprived)
| 5 (most deprived)
|Body mass index (per 5 kg/m2 increase)
|Any dependency prior to hospital admission
|Sedated for entire of the first 24 hr
|Highest temperature (per 1°C increase)
|Lowest systolic blood pressure (mm Hg)—RCS (78, 95, 121)
| Spline base variable 1
| Spline base variable 2
|Highest heart rate (per 10 beats/min increase)
|Highest respiratory rate (per 10 breaths/min increase)
2 (per 50 mm Hg increase)
|Mechanical ventilation (at Pao
2 = 300 mm Hg)
|Mechanical ventilation × Pao
2 (per 50 mm Hg increase)
|Highest blood lactate concentration (mmol/L)—RCS (0.9, 1.4, 2.5)
| Spline base variable 1
| Spline base variable 2
|Highest serum creatinine (µmol/L)—RCS (46, 72, 101, 327)
| Spline base variable 1
| Spline base variable 2
| Spline base variable 3
|Highest serum urea (mmol/L)—RCS (3.5, 7.0, 16.6)
| Spline base variable 1
| Spline base variable 2
|Lowest hemoglobin concentration (per 10 g/L increase)
|Lowest platelet count (×109/L)—RCS (134, 232, 375)
| Spline base variable 1
| Spline base variable 2
RCS = restricted cubic spline.
RCS (k1,…,kj) indicates restricted cubic spline with knots at positions k1 to kj, corresponding to the following base variables for prognostic factor x: spline base variable 1 = x; spline base variable i+1 = (max[(x – ki)3, 0] – (kj – ki) × max[(x – kj − 1)3, 0]/(kj– kj−1) + (kj – 1 – ki) × max[(x – kj)3, 0]/(kj –kj − 1))/(kj – k1)2; i =1, …, j – 2
Age was the strongest predictor of mortality with older patients significantly more likely to die. The association between age and mortality was nonlinear; the hazard ratio rose sharply after 60 years. Although the proportion of males critically ill with COVID-19 was much higher than that in the general population, once receiving critical care, there was no association between male sex and 30-day mortality after adjusting for other factors. The likelihood of dying within 30 days was 27% higher for Asian patients than White patients, although comparisons of White with other ethnic groups (Black and mixed/other) were not statistically significant after correcting for the other factors in the model. Patients who live in an area in among the most deprived fifth of locations in the UK were 14% more likely to die than those living in the least deprived areas. BMI showed a weak linear association with 30-day mortality, with a 3% increased likelihood of death associated with every 5 kg/m2 increase in BMI. Patients with any level of prior dependency had 42% greater likelihood of death within 30 days than those able to live without assistance in daily activities and immunocompromised patients had a 47% higher likelihood of death. Being sedated for the first 24 hours of critical care was associated with a 13% increased likelihood of death.
Highest HR and highest respiratory rate had significant linear associations with 30-day mortality, with increasing values associated with a greater likelihood of death within 30 days of the start of critical care. After adjusting for other variables, highest temperature and lowest hemoglobin were not significant. Lowest systolic blood pressure showed a significant nonlinear association with both hypo- and hypertensions associated with increased likelihood of death.
A significant interaction was observed between Pao2/Fio2 and mechanical ventilation. To give the interaction a meaningful interpretation, Pao2/Fio2 was centered at the threshold for defining mild acute respiratory distress syndrome (ARDS), 300 mm Hg (16). A ventilated patient with a Pao2/Fio2 of 300 mm Hg was almost twice as likely to die within 30 days than a nonventilated patient (HR = 1.89). The likelihood of dying within 30 days increased with decreasing values of Pao2/Fio2 for both ventilated and nonventilated patients, but the increase was steeper for nonventilated patients (HR = 1.35 per 50 mm Hg decrease) than for ventilated patients (HR = 1.21 per 50 mm Hg decrease). Mortality increased steeply with increasing lactate up to around 2 mmol/L after which the increase was more gradual.
Highest creatinine had a complex nonlinear association with mortality. Compared with the normal range, there was a small and nonsignificant increase in likelihood of 30-day mortality associated with low values of creatinine. Likelihood of mortality increased steeply with increasing creatinine above the normal range up to values around 200 µmol/L, after which the curve leveled off. Likelihood of death within 30 days increased with increasing values of highest urea up to 10 mmol/L, after which there was no further increase. Decreasing values of the lowest platelet count below the normal range were associated with an increased likelihood of death. This association flattened off at a platelet count of around 300 × 109/L, with no further change in the likelihood of death above this value.
Model Fit and Sensitivity Analyses
Harrell c-statistic for the model was 0.73, indicating acceptable discrimination. On graphical examination, the proportional hazards assumption appeared to be satisfied. There were no meaningful differences in the results using complete case data (eTable 4, Supplemental Digital Content 6, http://links.lww.com/CCM/F967).
We have identified demographic and clinical factors significantly associated with mortality within 30 days of the start of critical care among patients critically ill with confirmed COVID-19 and described their association with the outcome. Our results show a strong association of older age with mortality among critically ill patients. Asian ethnicity was also associated with worse outcome, with Black ethnicity demonstrating a weaker and nonsignificant association. Surprisingly, despite male patients being substantially overrepresented in the cohort of patients receiving critical care for COVID-19, there was no association with worse outcome once adjusted for patient characteristics. Among physiologic and laboratory variables, the strongest predictor was Pao2/Fio2; this is perhaps unsurprising, as reduced Pao2/Fio2 is the defining clinical characteristic for the severity of ARDS and hypoxemia is one of the main signs of severe COVID-19. As clinicians tend to adjust inspired oxygen fraction to achieve a desired Pao2 or Spo2, the Pao2/Fio2 ratio is mostly a function of inspired oxygen fraction rather than hypoxemia, other than in extreme cases where the desired oxygenation cannot be achieved. Thus, the association with mortality may not represent an association of hypoxemia per se with death so much as increased oxygen requirement as an indicator of lung or overall disease severity. The association between Pao2/Fio2 and mortality was stronger for nonventilated patients than for ventilated patients. This may reflect the additional control over the patient’s respiratory physiology offered by adjusting ventilator settings. We identified associations with increased creatinine and urea, suggesting deteriorating renal function was associated with higher mortality, as seen in many other forms of critical illness. As with other severe infections, an increasing blood lactate concentration was associated with an increasing risk of mortality. The association of low platelet counts and low creatinine concentrations with increased mortality may represent a confounding effect of unrecorded preexisting chronic disease or sarcopenia, as the values were recorded within 24 hours of the start of critical care and so would not have been influenced by treatment.
The main strengths of the study are the large, representative, and high-quality clinical dataset and the rigor of variable collection, with a coverage of 100% of adult general level 3 (intensive care) units in England, Wales, and Northern Ireland plus substantial numbers of specialist critical care units and temporary surge areas. There are, however, some limitations. The available predictors and outcomes were limited to those recorded in the Case Mix Programme, which were in turn driven by the need to ensure that data could be collected accurately in all participating units under current circumstances. Consequently, data were not available for some variables that have been found to be significant predictors of progression to severe disease or mortality in studies of hospitalized patients, for example, lactate dehydrogenase (17–21), C-reactive protein (17,21–25), and lymphocyte count (20–22,24,26) or neutrophil:lymphocyte ratio (18,25). The association of these variables with mortality could not be studied here. Recording of chronic health conditions in the Case Mix Programme is limited to only very severe comorbidities, and consequently, only small numbers of patients were reported to have comorbidities. As a result of this, only a combined variable corresponding to immunocompromise was included in the model. This does not preclude other chronic health conditions having important associations that we were not able to explore. It is also possible that the low levels of severe comorbidities reported may be an underestimate of the true rates, due to the difficulties of obtaining an accurate medical history in the pandemic circumstances. We also have no information on limitations of care, which may affect the treatments patients received and their subsequent outcomes.
The largest study to date to look specifically at prognostic factors among critically ill patients with COVID-19 reported on 3,988 patients admitted to critical care units in the Lombardy region of Italy (4). Although this study included more detailed information on comorbidities and identified a number of comorbidities associated with worse outcome (hypercholesterolemia, type 2 diabetes, and chronic obstructive pulmonary disease), the physiologic parameters considered were limited to only positive end-expiratory pressure, Fio2 and Pao2/Fio2, all of which had significant associations with mortality. In a study of 239 critically ill patients from Wuhan, China, the only laboratory variable found to predict mortality in a Cox proportional hazards model was platelet count (27).
This project concentrated solely on identifying key prognostic factors for poor outcome (28) and was not seeking a model with superior discrimination and calibration. Nonetheless, discrimination was worse than that has been reported for other models in a broader hospital setting (7). This may represent the inherent difficulties of prediction within a highly selected population with a narrower range of disease severity or the absence of important predictors from our model. Alternatively, it may reflect overfitting of the previous models due to small sample sizes with low numbers of events (7).
We do not recommend the application of our model for predictions at an individual level. However, given that the dataset was large and comprehensive, if validated, our results would provide useful insight into models for the prediction of outcome for patients admitted to critical care and enable identification of patients more likely to have a poor outcome. Importantly, none of the associations identified here should be interpreted as being causal. Not only are there likely to be other unmeasured factors that would confound any relationships between the prognostic factors evaluated here and mortality but also is there the act of selecting patients based on their requirement for critical care, likely representing “conditioning on a collider” (29). In this context, admission to critical care would be a collider for any prognostic factor associated with the decision to admit to critical care if there are other unmeasured factors that are the causes of both admission to critical care and mortality. However, this does not affect our ability to use these factors to make predictions of outcome among those patients admitted to critical care.
Our results show that factors that predict mortality within 30 days of the start of critical care for patients with confirmed COVID-19 include age; prior dependency; immunocompromise; patient characteristics including increasing BMI and Asian ethnicity; and physiologic and laboratory variables that indicate organ dysfunction particularly in the respiratory, cardiovascular, and renal systems. We encourage future research to extend these findings with the addition of inflammatory markers and lymphopenia. Further research may enable us to translate this work into a risk-prediction model, permitting benchmarking of critical care providers to aid learning ahead of any further wave of the pandemic.
We thank and respect all those working in critical care units across England, Wales, and Northern Ireland and contributing to the care of patients and, particularly, those responsible for submitting data rapidly and regularly during the emerging COVID-19 epidemic.
Additional Intensive Care National Audit & Research Centre Coronavirus Disease 2019 Team Members: Yemi Banjo, Kasia Borowczak, Tom Cousins, Peter Cummins, Keji Dalemo, Robert Darnell, Hanna Demissie, Laura Drikite, Andrew Fleming, Ditte Frederiksen, Sarah Furnell, Abdo Hussein, Abby Koelewyn, Tim Matthews, Izabella Orzechowska, Sam Peters, Alvin Richards-Belle, Tyrone Samuels, and Michelle Saull.
1. Wu Z, McGoogan JM. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: Summary of a report of 72 314 cases from the Chinese center for disease control and prevention. JAMA. 2020; 323:1239–1242
2. Azoulay E, Zafrani L, Mirouse A, et al. Clinical phenotypes of critically ill COVID-19 patients. Intensive Care Med. 2020; 46:1651–1652
3. Cummings MJ, Baldwin MR, Abrams D, et al. Epidemiology, clinical course, and outcomes of critically ill adults with COVID-19 in New York city: A prospective cohort study. Lancet. 2020; 395:1763–1770
4. Grasselli G, Greco M, Zanella A, et al. Risk factors associated with mortality among patients with COVID-19 in intensive care units in Lombardy, Italy. JAMA Intern Med. 2020; 180:1–11
5. Mitra AR, Fergusson NA, Lloyd-Smith E, et al. Baseline characteristics and outcomes of patients with COVID-19 admitted to intensive care units in Vancouver, Canada: A case series. CMAJ. 2020; 192:E694–E701
6. Yang X, Yu Y, Xu J, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: A single-centered, retrospective, observational study. Lancet Respir Med. 2020; 8:475–481
7. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal. BMJ. 2020; 369:m1328
8. Doidge JC, Mouncey PR, Thomas K, et al. Trends in intensive care for patients with COVID-19 in England, Wales and Northern Ireland. Preprints. 2020; 2020080267
9. Bartlett JW, Seaman SR, White IR, et al. Alzheimer’s Disease Neuroimaging Initiative*: Multiple imputation of covariates by fully conditional specification: Accommodating the substantive model. Stat Methods Med Res. 2015; 24:462–487
10. White IR, Royston P. Imputing missing covariate values for the Cox model. Stat Med. 2009; 28:1982–1998
11. Rubin DB: Multiple Imputation for Nonresponse in Surveys. 1987, New York, NY: J Wiley & Sons
12. Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing a clinical prediction model. BMJ. 2020; 368:m441
13. Harrell FE Jr: Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis. 2001, New York, NY: Springer
14. Schoenfeld D. Partial residuals for the proportional hazards. Biometrika. 1982; 69:239–241
15. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996; 15:361–387
16. Ranieri VM, Rubenfeld GD, Thompson BT, et al. Acute respiratory distress syndrome: The Berlin definition. JAMA. 2012; 307:2526–2533
17. Gong J, Ou J, Qiu X, et al. A tool to early predict severe 2019-novel coronavirus pneumonia (COVID-19): A multicenter study using the risk nomogram in Wuhan and Guangdong, China. medRxiv. 2020.03.17.20037515
18. Liang W, Liang H, Ou L, et al. China Medical Treatment Expert Group for COVID-19: Development and validation of a clinical risk score to predict the occurrence of critical illness in hospitalized patients with COVID-19. JAMA Intern Med. 2020; 180:1081–1089
19. Wu C, Chen X, Cai Y, et al. Risk factors associated with acute respiratory distress syndrome and death in patients with coronavirus disease 2019 pneumonia in Wuhan, China. JAMA Intern Med. 2020; 180:1–11
20. Xie J, Hungerford D, Chen H, et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv. 2020.03.28.20045997
21. Yan L, Zhang HT, Goncalves J, et al. A machine learning-based model for survival prediction in patients with severe COVID-19 infection. medRxiv. 2020.02.27.20028027
22. Fang C, Bai S, Chen Q, et al. Deep learning for predicting COVID-19 malignant progression. medRxiv. 2020.03.20.20037325
23. Lu J, Hu S, Fan R, et al. ACP risk grade: A simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. medRxiv. 2020.02.20.20025510
24. Petrilli CM, Jones SA, Yang J, et al. Factors associated with hospital admission and critical illness among 5279 people with coronavirus disease 2019 in New York city: Prospective cohort study. BMJ. 2020; 369:m1966
25. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: A retrospective cohort study. Lancet. 2020; 395:1054–1062
26. Tan L, Wang Q, Zhang D, et al. Lymphopenia predicts disease severity of COVID-19: A descriptive and predictive study. Signal Transduct Target Ther. 2020; 5:33
27. Xu J, Yang X, Yang L, et al. Clinical course and predictors of 60-day mortality in 239 critically ill patients with COVID-19: A multicenter retrospective study from Wuhan, China. Crit Care. 2020; 24:394
28. Riley RD, Hayden JA, Steyerberg EW, et al. PROGRESS Group: Prognosis research strategy (PROGRESS) 2: Prognostic factor research. PLoS Med. 2013; 10:e1001380
29. Cole SR, Platt RW, Schisterman EF, et al. Illustrating bias due to conditioning on a collider. Int J Epidemiol. 2010; 39:417–420