Secondary Logo

Journal Logo

Original Clinical Report

Rapid Development of a Tool for Prioritizing Patients with Coronavirus Disease 2019 for Intensive Care

Roy, Melyssa MB ChB, PhD1; Hansen, Paul PhD2,3; Sullivan, Trudy PhD1; Ombler, Franz PhD3; Kiore, Mathew BEd4; Stapleton, Andrew MB BS, FANZCA, FCICM5; Carr, Craig MB ChB, FRCA, FFICM6

Author Information
Critical Care Explorations: March 2021 - Volume 3 - Issue 3 - p e0368
doi: 10.1097/CCE.0000000000000368


The coronavirus disease 2019 (COVID-19) pandemic has generated unprecedented demand for intensive care beds and ventilators in health systems around the world. The disease has a 2–7% mortality rate, with up to 20% of patients requiring hospitalization, of which 4–14% are admitted to intensive care with 60–88% receiving invasive ventilatory support (1–6). ICUs in China, Italy, the United Kingdom, and the United States have been overwhelmed, with ICUs in other countries similarly threatened (7). This pressure on ICUs has resulted in patients needing to be prioritized for admission. Valid, transparent, and pragmatic decision-support tools to help clinicians with these ethically difficult, often “life or death” decisions, are required (8,9).

Prioritizing patients for ICU involves clinicians weighing up each patient’s likely benefit from treatment and survival, by considering the severity of illness and comorbid conditions, as well as other factors (10–13). Because COVID-19 patients can require long periods of mechanical ventilatory support (median 13 vs 9 days for noncoronavirus disease [non-COVID] ICU patients) (6), a patient’s clinical assessment should also be compared with other patients’ opportunities to benefit. Mortality rates of invasively ventilated COVID-19 patients are high—almost double the rate of patients with non-COVID viral pneumonia (40% vs 21%) (6). Initiation of futile treatment likely to further distress patients in their final days is inappropriate. Judgments about withdrawing critical support from one patient in favor of another (with the objective of maximizing the number of patients successfully treated) are even more fraught.

Prioritizing patients with the best prognoses is underpinned by utilitarian ethical principles associated with maximizing lives or life years saved (9,14,15). The “Fair Innings” principle is also commonly applied: younger patients are prioritized over older patients, so that everyone has the opportunity for a “fair innings” in life (16). These principles can conflict. Is it better to ventilate a multimorbid 30-year-old with a poor prognosis and long length of stay, or an otherwise well 60-year-old with a substantially lower risk of mortality and length of stay? How do clinicians (and ethicists) resolve such difficult decisions, and what tools can support them?

Pragmatic considerations of staff workload and other essential therapy components are also important when ICUs are under pressure (17,18). Medically complex and vulnerable patients—such as those needing renal replacement therapies and patients at both extremes of body mass index (BMI)—require more expert nursing. The capacity to benefit from ventilation may be diminished by inadequate workforce expertise, essential equipment, or consumables.

Should healthcare workers who become infected with COVID-19 be prioritized because of their important and sometimes irreplaceable role in the pandemic response? “Essential workers”—however designated—may be prioritized based on the ethical principle of reciprocity in recognition of the extra risks they take to help others (9,14). In many countries, ICU workforces were already severely constrained before the pandemic; losing staff would curtail capacity further.

In New Zealand, where the current study is set, Māori (the indigenous population) and Pacific people are significantly disadvantaged in terms of health inequities and are likely to be disproportionately harmed by a COVID-19 outbreak. The World Health Organization promotes actions in pursuit of equity, regarding health as a human right (19), and in New Zealand, the Treaty of Waitangi (a constitutional document) requires government to achieve equity for Māori (20). Analogous considerations are relevant for other countries.

As the COVID-19 pandemic emerged globally in early 2020, a simple prioritization framework developed by White and Lo (21), adapted from an earlier framework (22), was adopted by many ICUs in the United States (23). Patients are prioritized on their likelihood to benefit, assessed by their Sequential Organ Failure Assessment (SOFA) score (24) and prognosis for near-term survival. Both criteria have four levels that are equally weighted, representing identical relative importance; thus, patients are prioritized according to their score in the range of 1–8 points. Pandemic healthcare workers receive additional priority, and if patients are prioritized equally, younger patients are selected (21).

Tyrrell et al (25) systematically reviewed another eight “guidelines” for prioritizing COVID-19 patients for ICU. Most are based primarily on patients’ likelihood of survival, with the objective of maximizing lives saved, although several are not explicit about their ethical principles. Most of these guidelines ignore the other prioritization considerations discussed earlier (e.g., staff workload and prioritization of essential workers). Tyrrell et al (25) concluded there is “a lack of high-quality evidence and guidelines on resource allocation during the pandemic” (p. 1).

Other COVID-19 ICU prioritization tools (also variously known as “protocols,” “algorithms,” and “schemes”) have also been developed, each with their own strengths and weaknesses. The triage protocol by Maves et al (26) includes a mortality prediction tool but does not recommend a particular one for use. The algorithm by Sprung et al (27), although relatively easy to implement due to its simplicity, has the disadvantage of sorting patients into just four prioritization categories, requiring tie-breakers based first on patients’ predicted life years saved and then “first-come, first-served”—an ethical principle rejected by most ethicists (8,9). The prioritization scheme by LeClerc et al (28) has separate pathways for how prioritization is implemented depending on the pressure ICUs are under, and although patients may be excluded based on clinical frailty scores, preexisting conditions, and acute conditions, prioritization is still based on just SOFA score or occupational exposure. These shortcomings are consistent with the recommendation by Tyrrell et al (25), based on their abovementioned review, that “[f]uture guidelines need to be evidence based and developed using robust methodologies” (p. 9). The aim of the present article is to contribute to this objective.

As the pandemic began to threaten New Zealand in February 2020, it was immediately apparent that because the country had only 4.6 ICU beds per 100,000 population—for example, compared with 12.5 in Italy and 29.4 in the United States—a COVID-19 prevalence of just 0.4% would exceed New Zealand’s ICU capacity. The Australian and New Zealand Intensive Care Society emphasized that the allocation of intensive care resources must be “consistent, transparent, objective and ethical” (p. 98) (12). The development of a prioritization tool based on valid and reliable criteria and weights if ICUs were to become overwhelmed was identified as a vital preparation.

This article reports on a new approach implemented in New Zealand for rapidly developing a prioritization tool based on methods from multi-criteria decision analysis (MCDA)—a decision-making methodology when multiple criteria need to be considered together (29). The tool was developed over a period of just 10 days as the country was preparing to go into national lockdown, with the tool ready for use on 26 March—27 days after New Zealand’s first reported COVID-19 case and 1 day after lockdown (30). Fortunately for New Zealand, the tool has not been used because to date public health measures have successfully eliminated the disease from New Zealand. Despite it not being possible to report on the tool’s performance “in the field” (ICU), the article’s primary objective is to explain the approach, supported by a demonstration of its feasibility for rapid implementation.


Participants and Ethics Approval

On 17 March 2020, an expert group for developing the prioritization tool was convened. The group comprised representatives from intensive care medicine (including the sixth and seventh authors), Māori health (the fifth author), intensive care nursing, infectious diseases, neonatology, and ethics. The group’s work was supported by the first four authors—representing public health medicine, health economics, and decision analysis—and overseen by the chairperson of the New Zealand National Ethics Advisory Committee and a senior representative from the New Zealand Ministry of Health. Ethics approval was provided by the University of Otago Human Ethics Committee (reference D19/071).

Inclusion and Exclusion Criteria

The inclusion and exclusion criteria from the Canadian pandemic triage protocol (31) were adapted to screen patients who could potentially benefit from ICU treatment.


MCDA is increasingly used for prioritization in the health sector (32), such as for determining access to elective services (33) and developing national guidelines for allocating pandemic influenza vaccines (34). The methodology is intended to reduce biases from decision-makers relying on their “clinical instinct,” resulting in more transparent and consistent decisions (35).

Two elements are fundamental to MCDA in the present context. First is the specification of criteria for prioritizing COVID-19 patients for ICU and levels of severity within each criterion. Second is the determination of weights on the criteria and levels, representing their relative importance. The resulting prioritization tool (criteria and weights) is implemented by rating each patient on the criteria and then summing the weights to produce a score. Patients are ranked (prioritized) by their scores.

Specialized MCDA software, 1000minds (, which implements the “Potentially All Pairwise RanKings of all possible Alternatives” (PAPRIKA) method to determine the weights (36), was used to support the group’s work. This software and method have been used since 2004 in a wide range of health applications, including for prioritizing patients for elective services (33), antibiotic-resistant diseases for research into new drugs (37), and noncritical COVID-19 patients for hospital admission (38) (at about the same time as the present project).

Prioritization Tool

The process for creating the prioritization tool involves eliciting and codifying the experts’ clinical knowledge and preferences. As well as their training, the experts have tacit knowledge based on their clinical practice and networks and are up-to-date with the emerging COVID-19 literature and in touch with international colleagues on the COVID-19 frontline. The process has four stages, as outlined below.

Patient Vignette Rankings.

To enable identification of relevant criteria for prioritizing COVID-19 patients for ICU, the experts were asked to use their clinical judgment to prioritize 10 patient case “vignettes” representative of people likely to be considered for admission (Supplementary Table 1, Each vignette is a concise description of patient information, including demographic characteristics, salient medical history, and clinical presentation. The experts ranked the vignettes individually and then by consensus as a group, which promoted discussion about relevant prioritization criteria and validation of clinical judgments within the group.

Criteria and Levels.

The vignette ranking exercise and ensuing discussion resulted in the identification of clinically and ethically relevant criteria for prioritizing COVID-19 patients. The levels of severity within each criterion were also defined.


The PAPRIKA method (36) (for determining the weights on the criteria and their levels) involved the experts voting on a series of simple questions based on choosing which of two hypothetical patients should be prioritized for ICU. To minimize the experts’ cognitive burden, the hypothetical patients in each question were defined on just two criteria at a time, with the other criteria assumed to be identical for both patients (Fig. 1).

Figure 1.
Figure 1.:
Example of a trade-off question. BMI = body mass index, COVID-19 = coronavirus disease 2019.

Fundamental to the questions is that the two criteria included in each question are specified so that the experts had to confront a trade-off when prioritizing the two patients, thereby revealing the relative importance of the criteria. If the experts’ votes for a trade-off question were not unanimous, there was a discussion until a consensus or clear majority (> 80%) was reached. From the answers to the questions, the method calculated the weights on the criteria and levels.

Reliability and Validity.

The test-retest reliability of the experts’ answers was evaluated by repeating some of the trade-off questions and voting on them again. To evaluate face validity, the experts considered the intuitive plausibility of the relative importance of the criteria and levels implied by the weights.

As a further validity test, the experts’ consensus ranking of the 10 vignettes from stage 1 was used as a “pseudo-gold standard” to compare the ranking produced by the tool. Finally, a new sample of COVID-19 patient cases was used to check that they were appropriately differentiated with respect to the distribution of their scores from the tool.


Inclusion and Exclusion Criteria

The inclusion and exclusion criteria are reported in Supplementary Table 2 ( These criteria apply to adult COVID-19 patients (≥ 18 yr) who could consent to and potentially benefit from intensive care and for whom such care is appropriate.

Prioritization Tool

Twelve experts participated in the patient vignette ranking exercise and discussion, which resulted in the specification of eight clinical criteria and two additional “social” considerations for prioritizing patients. Classifications of common preexisting conditions into appropriate levels are reported in Table 1; existing disease classification systems were included for ease of use and to facilitate shared understanding among the wider healthcare community.

TABLE 1. - Examples of Diseases Corresponding to Levels Within Each Criterion
Criterion Level Examples
Preexisting cardiovascular conditions Significant Heart failure class II, severe hypertension > 180/120, severe valvular disease, ischemic heart disease; angina class II–III
Moderate Class 1 heart failure, atrial fibrillation, poorly controlled hypertension (> 160/110). Prior coronary percutaneous intervention
Mild Well-controlled hypertension
Preexisting respiratory conditions Significant FEV1 30–40% predicted, severe COPD
Moderate FEV1 40–80% predicted, moderate asthma/COPD, heavy smoker (> 20/d)
Mild FEV1 > 80% predicted, mild asthma
Other relevant medical conditions: renal, endocrine, neuromuscular, malignancy Moderate/significant Neuromuscular disease with respiratory impairment, metastatic malignancy treated with palliative intent, stage 4–5 chronic kidney disease, diabetes with end-organ damage
Mild Diabetes without end-organ damage, stage 2–3 chronic kidney disease, malignancy managed with long-term stability or curative intent
Immunocompromised Moderate/significant Chemotherapeutic or posttransplant medications with significant immunocompromize, long-term high dose prednisone
Mild Inhaled steroids, low-dose steroids
COPD = chronic obstructive pulmonary disease, FEV1 = forced expiratory volume during first second.

The weights for the clinical criteria and their levels were determined in three meetings: a pilot-test of the voting exercise, which resulted in the criteria and their levels being refined in a second meeting, and then a final voting and consensus exercise. In the final meeting, which lasted approximately 2 hours, the experts voted on 49 trade-off questions, mostly resulting in a unanimous vote or a clear majority, with discussions for the remaining questions until consensus was reached.

The experts’ answers exhibited test-retest reliability, and the ranking of the vignettes produced by the tool matched the experts’ consensus ranking (the “pseudo-gold standard”). Overall, the experts were satisfied with the validity of the criteria and weights.

The criteria and weights are reported in Table 2. To implement the prioritization tool, each patient is rated on the eight criteria, and the corresponding weights are summed to produce a score (in the range 0–100%). As an illustration, the tool’s ranking of the 10 vignettes is reported in Supplementary Table 3 (

TABLE 2. - Criteria and Weights for ICU Prioritization
Criterion/Levels Weight, %
Sequential Organ Failure Assessment score
 > 11 0
 ≤ 7 or single organ failure 15.0
 8–11 15.7
Preexisting cardiovascular conditions
 Significant 0
 Moderate 9.2
 None/mild 15.7
Functional capacity
 Poor: MEWS < 5 0
 Normal: MEWS > 5 15.7
Age (independent of other criteria), yr
 > 75 0
 60–75 4.6
 40–59 6.5
 19–39 11.1
 ≤ 18 12.4
Preexisting respiratory conditions
 Significant 0
 Moderate 4.6
 Mild 9.2
 None 11.1
 Moderate/significant 0
 None/mild 11.1
Other relevant medical conditions
 Moderate/significant 0
 None/mild 9.2
Body mass index (independent of comorbidity and function)
 > 50 0
 40–50 4.6
 < 18 6.5
 19–40 9.2
MEWS = modified early warning score.
The bolded values represent the relative weights of the criteria overall (i.e., bolded values sum to 100%). The expert group considered that patients with Sequential Organ Failure Assessment score < 7 were less likely to require admission to ICU but may do so in the event of isolated respiratory failure. The ranking of the levels and their weights may change in future revisions of the tool.

Social Equity Considerations

The COVID-19 pandemic poses a disproportionate risk to marginalized populations, including Māori and Pacific people of New Zealand who experience poorer health outcomes significantly contributed to by the effects of colonization (39,40). Therefore, the expert group, endorsed by the representatives from ethics and the Ministry of Health, agreed that Māori and Pacific people should have some degree of higher priority. It was also deemed that healthcare workers and other critical frontline personnel who become infected with COVID-19 while working should have higher priority. The healthcare workers in the expert group recused themselves from this specific discussion, although they contributed to the wider dialogue.

These two social equity considerations were initially intended to be incorporated with the eight clinical criteria within the tool. However, when the voting exercise was being pilot-tested, it soon became apparent that the experts found thinking about possible trade-offs between social equity considerations and clinical criteria cognitively difficult. Furthermore, doctors’ respect for the principles of the Hippocratic Oath may prevent them from differentiating between patients based on ethnicity or occupation, which, in practice, may result in the social equity considerations receiving no weight at all. A pragmatic solution was to separate the social equity considerations and evaluate them alongside the clinical prioritization tool.

Thus, the prioritization tool involves rating patients on the clinical criteria—generating a score in the range 0–100%—and collecting supporting information about patients with respect to the abovementioned social equity considerations. The score, representing clinical likelihood of benefit, as well as social equity considerations and other supporting information, can then be assessed on a case-by-case basis by a panel of three ICU experts within the overarching decision framework.

Further work is being undertaken by experts in Māori health and ethics to effectively integrate social equity concerns within the prioritization tool. One approach suggested is to implement a reserve system: a designated proportion of ICU beds could be reserved for Māori and Pacific people in proportion to their percentage of the population and augmented for their disparity in health outcomes due to greater prevalence of comorbid conditions.


Intensive care medicine involves complex, rapid decision-making that usually depends on individual clinical judgment (11). In “normal” times, when resources are relatively unconstrained, clinicians usually commence a “trial of ICU” when a patient’s potential to benefit is uncertain. In contrast, during a major pandemic, when resources are severely constrained and not all patients can be treated, rationing ICU resources is inevitable. Prioritization tools support clinicians in making these difficult decisions and reduce the associated burden and distress.

Evidence about prognostic clinical factors in COVID-19 patients is accumulating, but as successful treatment strategies are identified, survival likelihood is continually changing. This variability limits the reliability of mortality calculators developed from historical datasets. Arguably, clinical decision-making is still best guided by expert judgment informed by available evidence—as summarized in a tool like the one developed here. Alternative approaches include the use of machine learning or other data-driven algorithms, but this is problematic due to issues with biases in case reporting (41), which increases the likelihood of errors compounding the effects of socioeconomic disparities (42).

Although based on existing pandemic triage tools (31), the proposed prioritization tool for New Zealand was developed specifically for prioritizing COVID-19 patients for admission to ICUs. It represents the consensus of a group of intensive care experts about valid and reliable criteria and weights for assessing patients’ likely clinical benefit. Social equity considerations can also be incorporated within the overarching decision framework.

One limitation is that this proposed tool has not been used in practice yet—because, as mentioned earlier, public health measures have successfully controlled the pandemic in New Zealand so far—and so it has not been possible to evaluate the tool’s performance in practice. Nonetheless, the main purpose of reporting on the New Zealand application here was to demonstrate the feasibility of the MCDA methods explained in the article for rapidly developing a prioritization tool, which could be used in other countries too.

A new tool can be created from scratch in a matter of just a few days, or a tool such as the one presented here could be adapted or updated in just a few hours. Because all the data from the process are stored electronically by the 1000minds MCDA software, the criteria and weights can be easily revised without repeating the whole process. As new evidence emerges, further criteria can be added or obsolete criteria deleted without involving criteria unaffected by such changes. If clinical judgments change about the relative importance of the criteria, clinicians are required to answer only the affected trade-off questions at stage 3 of the process outlined earlier, resulting in revised weights on the criteria. For example, if COVID-19 were to threaten to overwhelm New Zealand’s ICUs again, the method could be rapidly reapplied to update the tool to incorporate the latest evidence about the disease.

Obvious possible refinements to the current tool include replacing the “functional capacity” criterion with clinical frailty scores, reviewing the levels defined for the SOFA criterion, and including other factors that are prognostic of a poor outcome, such as the presence of lymphopenia. Within the social equity considerations, other at-risk groups could potentially be included. Further work is needed to determine how this tool would be used within an overarching decision framework to address these issues of social equity.

Additional work would also be required to extend the tool to triage COVID-19 patients alongside other ICU patients. Many of the criteria are helpful for assessing any critically ill patient, but that is not the purpose of the tool. Clinicians already assess and triage the general ICU patient population on a regular basis. The tool is designed to assist within a wider decision-making setting to manage an unfamiliar disease threatening to overwhelm ICUs.

Although other COVID-19 ICU prioritization tools are available (e.g., the framework by White and Lo [21]), they focus mainly on isolated clinical factors such SOFA score and prognosis for near-term survival and do not adequately consider all relevant clinical factors (also, the validity of equal weights is moot). Factors such as BMI, functional status, and the presence and severity of comorbid conditions clearly affect outcomes differentially, and so they should be included and accurately weighted to assess a patient’s likely clinical benefit (6). These additional considerations were able to be included in the proposed tool in large part because of the PAPRIKA method used for determining the weights on the criteria and levels.

Fundamental to PAPRIKA’s usefulness was the cognitive ease with which the participating experts were able to answer PAPRIKA’s trade-off questions (involving repeatedly choosing between two hypothetical patients defined on two criteria at a time with respect to who should be admitted first). Also, with just three possible answers (Fig. 1), PAPRIKA’s questions are suited to rapid group decision-making via voting and consensus.

The resulting weights—codifying the experts’ clinical knowledge and preferences—are likely to be more valid and reliable than weights produced by other MCDA methods, which usually involve more cognitively difficult elicitation techniques based on ratio-scale measurements of preferences (also making them less amenable to group decision-making). “The advantage of choice-based methods is that choosing, unlike scaling, is a natural human task at which we all have considerable experience, and furthermore it is observable and verifiable” (43). PAPRIKA also enables the criteria and weights in the prioritization tool to be easily revised as new evidence emerges.

The approach explained and demonstrated in this article could be used in other countries whose ICUs are at risk of being overwhelmed by the COVID-19 pandemic to rapidly develop their own prioritization tools—in effect, adapting the proposed tool for New Zealand to other settings. In the event that future crises threaten to overload ICUs, other prioritization tools could also be rapidly developed.


We thank the Coronavirus Disease 2019 ICU Multicriteria Decision Analysis Working Group: Craig Carr, Andrew Stapleton, Gillian Bishop, Seton Henderson, Liza Edmonds, Brendon Arnold, Pam Adams, Elaine Fernandes, Mathew Kiore, Gilbert Taurua. We also thank Neil Pickering and Chris McEwan.


1. Richardson S, Hirsch JS, Narasimhan M, et al.; the Northwell COVID-19 Research Consortium. Presenting characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York city area. JAMA. 2020; 323:2052–2059
2. Livingston E, Bucher K. Coronavirus disease 2019 (COVID-19) in Italy. JAMA. 2020; 323:1335
3. Grasselli G, Zangrillo A, Zanella A, et al.; COVID-19 Lombardy ICU Network. Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. JAMA. 2020; 323:1574–1581
4. Yang X, Yu Y, Xu J, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: A single-centered, retrospective, observational study. Lancet Respir Med. 2020; 8:475–481
5. Wu Z, McGoogan JM. Characteristics of and important lessons from the Coronavirus disease 2019 (COVID-19) outbreak in China: Summary of a report of 72 314 cases from the Chinese center for disease control and prevention. JAMA. 2020; 323:1239–1242
6. Intensive Care National Audit & Research Centre. ICNARC Report on COVID-19 in Critical Care – 24 July 2020. Intensive Care National Audit & Research Centre1–40
7. Rosenbaum L. Facing Covid-19 in Italy - ethics, logistics, and therapeutics on the epidemic’s front line. N Engl J Med. 2020; 382:1873–1875
8. Truog RD, Mitchell C, Daley GQ. The toughest triage - allocating ventilators in a pandemic. N Engl J Med. 2020; 382:1973–1975
9. Emanuel EJ, Persad G, Upshur R, et al. Fair allocation of scarce medical resources in the time of Covid-19. N Engl J Med. 2020; 382:2049–2055
10. Vergano M, Bertolini G, Giannini A, et al. Crit Care. 2020; 24:165
11. James FR, Power N, Laha S. Decision-making in intensive care medicine - A review. J Intensive Care Soc. 2018; 19:247–258
12. Warrillow S, Austin D, Cheung W, et al. ANZICS guiding principles for complex decision making during the COVID-19 pandemic. Crit Care Resusc. 2020; 22:98–102
13. British Medical Association. Covid-19 – Ethical Issues. A Guidance Note. 2020. Available at: Accessed November 1, 2020
14. Rosenbaum S. Ethical Considerations for Decision Making Regarding Allocation of Mechanical Ventilators During a Severe Influenza Pandemic or Other Public Health Emergency. 2011, Atlanta, GA: Centers of Disease Control and Prevention
15. Wilkinson D. ICU triage in an impending crisis: Uncertainty, pre-emption and preparation. J Med Ethics. 2020; 46:287–288
16. Archard D, Caplan A. Is it wrong to prioritise younger patients with Covid-19? BMJ. 2020; 369:m1509
17. Meares HD, Jones MP. When a system breaks: Queueing theory model of intensive care bed needs during the COVID-19 pandemic. Med J Aust. 2020; 212:470–471
18. Noronha KVMS, Guedes GR, Turra CM, et al. The COVID-19 pandemic in Brazil: Analysis of supply and demand of hospital and ICU beds and mechanical ventilators under different scenarios. Cad Saude Publica. 2020; 36:e00115320
19. Hosseinpoor AR, Bergen N, Schlotheuber A. Promoting health equity: WHO health inequality monitoring at global and national levels. Glob Health Action. 2015; 8:29034
20. New Zealand Ministry of Health. Achieving Equity in Health Outcomes. Summary of a Discovery Process. 2019, Wellington, NZ: Ministry of Health
21. White DB, Lo B. A framework for rationing ventilators and critical care beds during the COVID-19 pandemic. JAMA. 2020; 323:1773–1774
22. White DB, Katz MH, Luce JM, et al. Who should receive life support during a public health emergency? Using ethical principles to improve allocation decisions. Ann Intern Med. 2009; 150:132–138
23. de Puy Kamp M, Devine C, Griffin D. As coronavirus cases grow, hospitals adopt a system to rank patients for treatment. CNN Investigates. 2020. Available at: Accessed November 1, 2020
24. Ferreira FL, Bota DP, Bross A, et al. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001; 286:1754–1758
25. Tyrrell CSB, Mytton OT, Gentry SV, et al. Managing intensive care admissions when there are not enough beds during the COVID-19 pandemic: A systematic review. Thorax. 2021; 76:302–312
26. Maves RC, Downar J, Dichter JR, et al.; ACCP Task Force for Mass Critical Care. Triage of scarce critical care resources in COVID-19 an implementation guide for regional allocation: An expert panel report of the Task Force for Mass Critical Care and the American College of Chest Physicians. Chest. 2020; 158:212–225
27. Sprung CL, Joynt GM, Christian MD, et al. Adult ICU triage during the coronavirus disease 2019 pandemic: Who will live and who will die? Recommendations to improve survival. Crit Care Med. 2020; 48:1196–1202
28. Leclerc T, Donat N, Donat A, et al. Prioritisation of ICU treatments for critically ill patients in a COVID-19 pandemic with scarce resources. Anaesth Crit Care Pain Med. 2020; 39:333–339
29. Stewart T, Belton V. Multiple Criteria Decision Analysis: An Integrated Approach. 2002, Dordrecht, NL: Kluwer
30. New Zealand Ministry of Health. Covid-19 Current Cases. 2020. Available at: Accessed November 1, 2020
31. Christian MD, Hawryluck L, Wax RS, et al. Development of a triage protocol for critical care during an influenza pandemic. CMAJ. 2006; 175:1377–1381
32. Thokala P, Devlin N, Marsh K, et al. Multiple criteria decision analysis for health care decision making – An introduction: Report 1 of the ISPOR MCDA emerging good practices task force. Value Heal. 2016; 19:1–13
33. Hansen P, Hendry A, Naden R, et al. A new process for creating points systems for prioritising patients for elective health services. Clin Gov. 2012; 17:200–209
34. US Department of Health and Human Services, US Department of Homeland Security. Guidance on Allocating and Targeting Pandemic Influenza Vaccine. 2018. Available at: Accessed November 1, 2020
35. Kahneman D. Thinking, Fast and Slow. 2011, New York, NY: Farrar, Straus and Giroux
36. Hansen P, Ombler F. A new method for scoring additive multi-attribute value models using pairwise rankings of alternatives. J Multi-Criteria Decis Anal. 2008; 15:87–107
37. Tacconelli E, Carrara E, Savoldi A, et al.; WHO Pathogens Priority List Working Group. Discovery, research, and development of new antibiotics: The WHO priority list of antibiotic-resistant bacteria and tuberculosis. Lancet Infect Dis. 2018; 18:318–327
38. De Nardo P, Gentilotti E, Mazzaferri F, et al.; members of the COVID-19MCDA Group. Multi-criteria decision analysis to prioritize hospital admission of patients affected by COVID-19 in low-resource settings with hospital-bed shortage. Int J Infect Dis. 2020; 98:494–500
39. Health Quality & Safety Commission New Zealand. A Window on the Quality of Aotearoa New Zealand’s Health Care 2019: A View on Māori Health Equity. 2019, Wellington, NZ: Health Quality & Safety Commission New Zealand
40. Verrall A, Norton K, Rooker S, et al. Hospitalizations for pandemic (H1N1) 2009 among Maori and Pacific Islanders, New Zealand. Emerg Infect Dis. 2010; 16:100–102
41. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prognosis of covid-19 infection: Systematic review and critical appraisal. BMJ. 2020; 369:m1328
42. Gianfrancesco MA, Tamang S, Yazdany J, et al. Potential biases in machine learning algorithms using electronic health record data. JAMA Intern Med. 2018; 178:1544–1547
43. Rummond MF, Sculpher MJ, Claxton K, et al. Methods for the Economic Evaluation of Health Care Programmes. 2015, Oxford, United Kingdom: Oxford University Press

coronavirus disease 2019; critical care; intensive care units; medical; multi-criteria decision analysis

Supplemental Digital Content

Copyright © 2021 The Authors. Published by Wolters Kluwer Health, Inc. on behalf of the Society of Critical Care Medicine.