Establishing a core outcomes set for massive transfusion: An Eastern Association for the Surgery of Trauma modified Delphi method consensus study

Abstract

BACKGROUND 

The management of severe hemorrhage has changed significantly over recent decades, resulting in a heterogeneous description of diagnosis, treatment, and outcomes in the literature, which is not suitable for data pooling. Therefore, we sought to develop a core outcome set (COS) to help guide future massive transfusion (MT) research and overcome the challenge of heterogeneous outcomes reporting.

METHODS 

Massive transfusion content experts were invited to participate in a modified Delphi study. For Round 1, participants submitted a list of proposed core outcomes. In subsequent rounds, panelists used a 9-point Likert scale to score proposed outcomes for importance. Core outcomes consensus was defined as >85% of scores receiving 7 to 9 and <15% of scores receiving 1 to 3. Feedback and aggregate data were shared between rounds.

RESULTS 

From an initial panel of 16 experts, 12 (75%) completed three rounds of deliberation to reevaluate variables not achieving predefined consensus criteria. A total of 64 items were considered, with 4 items achieving consensus for inclusion as core outcomes: blood products received in the first 6 hours, 6-hour mortality, time to mortality, and 24-hour mortality.

CONCLUSION 

Through an iterative survey consensus process, content experts have defined a COS to guide future MT research. This COS will be a valuable tool for researchers seeking to perform new MT research and will allow future trials to generate data that can be used in pooled analyses with enhanced statistical power.

LEVEL OF EVIDENCE 

Diagnostic Test or Criteria; Level V.

In Brief

@notoriousRBGmd; @seberly; @jnahmias1; @DanteYeh; @MarkusZiesmann; @UABTraumaModified Delphi method was used to achieve consensus on a Core Outcome Set for Massive Transfusion to improve consistency in reporting of outcome metrics in future trials.

Hemorrhage is a major preventable cause of death after traumatic injury; however, the management of uncontrolled massive hemorrhage in severely injured trauma patients can be a significant challenge. In the early stages of severe trauma, tissue damage and shock contribute to an imbalance between clotting, anticoagulation, and fibrinolysis.^1–3 These hemostatic abnormalities, exacerbated by the “lethal triad” of coagulopathy, hypothermia, and acidosis, can lead to worsening of uncontrolled bleeding. Management priorities include stopping blood loss and restoring circulating blood volume and hemostatic competence to reverse this state of shock and trauma-induced coagulopathy.

The definition of massive transfusion (MT) is somewhat controversial but was classically considered transfusion of 10 U of blood or more in a 24-hour period, as defined during the Vietnam War.³ The Joint United Kingdom Blood Transfusion and Tissue Transplantation Services defines MT as a loss of greater than one entire blood volume in 24 hours.⁴ These historical definitions are often criticized because they have never been validated as markers of bleeding severity, do not account for early deaths, and may lead to survivor bias.³ In addition, they are retrospective definitions and therefore not useful for patient management. Given that the median time to death from hemorrhage is less than 3 hours, more recent definitions of MT use shorter time intervals, such as more than 4 U of red blood cells in the first hour,⁵ or replacement of more than 50% of circulating blood volume in less than 3 hours.⁶ The critical administration threshold, defined as 3 U of blood or more in the first hour, has also been found to predict mortality more accurately than previous definitions, especially when critical administration threshold is met more than once.^7,8

There is a need for high-quality randomized controlled trials and meta-analyses to further advance our understanding and develop evidence-based guidelines related to hemorrhage control. Developing a core outcome set (COS) to help define valuable consensus endpoints will facilitate data pooling for systematic reviews and meta-analyses.⁹ These core outcomes are a minimum standard set of outcomes for future studies on this topic. Implementation of a COS maximizes the number of studies that can be compared for data pooling, without imposing a limit on additional outcomes of interest.

Thus, the Eastern Association for the Surgery of Trauma (EAST) created a COS task force to identify and develop COS guidelines in high-value areas. The initial COS development was driven by a Delphi survey to determine the topics of highest priority. Core outcome sets have already been developed for resuscitative endovascular balloon occlusion of the aorta¹⁰ and damage control laparotomy.¹¹ Massive transfusion was also identified as a high-priority topic, which would benefit from development of a COS.

PATIENTS AND METHODS

This MT COS study was developed following the Core Outcome Measures in Effectiveness Trials tool and conducted according to established standards described by the Core Outcome Set Standards for Development and Reporting.^12,13 The 18-item Core Outcome Set Standards for Development and Reporting checklist presented in Supplemental Digital Content, Supplementary Data 1, https://links.lww.com/TA/C848, covers the minimum reporting requirements related to the background, scope, methods, results, conclusion, and limitations of such studies.¹³ The study was registered with the Core Outcome Measures in Effectiveness Trials database¹⁴ and carried out under the oversight of the EAST COS Task Force.¹⁵ This study was deemed exempt by our institutional review board, and informed consent was assumed if a participant responded to the Delphi survey.

The EAST COS Task Force first reviewed a collection of high-impact peer-reviewed publications on MT in the EAST Landmark Papers collection.¹⁶ The first and last authors of these papers were invited to participate in the survey as expert panelists if they remained currently clinically active in the field of trauma or MT research. Task force members and identified experts were invited to nominate additional peers with known academic and clinical expertise in MT. From 31 initial invitations, 16 international (3 countries) and multidisciplinary experts in trauma and transfusion medicine comprised the final panel. This included a biostatistician who performed the intraclass correlation (ICC) analyses.

An effort was made to identify patients from support groups or other organizations for inclusion in the panel. Unfortunately, given the nature of MT, no such support groups exist. We felt that reaching out to patient communities through social media would introduce bias, as response rates through social media are historically low and would limit the range of patients because of self-selection. Therefore, patients were not included in this particular COS Delphi.

In Round 1, participants were asked to submit a free-form list of proposed core outcomes for further consideration. These were thematically grouped by the MT COS task force, and duplicate suggestions were removed. In Round 2 and thereafter, panelists were requested to rank each outcome according to a Likert scale ranging from 1 to 9, based on the Grading of Recommendations Assessment, Development, and Evaluation scale.¹⁷ Outcomes were presented to panelists in a random order to minimize bias. Scores of 1 to 3 represented less important outcomes, and scores of 7 to 9 represented critically important outcomes. We defined a core outcome a priori as any item for which >85% of panelists selected a score in the range of 7 to 9 and <15% of panelists selected a score in the range of 1 to 3. The letters to the experts explaining each round of the Delphi process have been included as Supplemental Digital Content, Supplementary Data 2, https://links.lww.com/TA/C849.

Acceptance that consensus has been reached for an outcome to be included in the COS requires agreement by the majority regarding the critical importance of that outcome. While our threshold of 85% was a slightly different range than those reported in prior COS publications,^18,19 there are no universally agreed consensus criteria in Delphi surveys and examples vary widely. According to Williamson et al.,⁹ the criteria used to determine when consensus has been reached are quite subjective but should be defined in advance to avoid biasing conclusions toward the researchers' own beliefs. We felt that using more rigorous predefined criteria would limit the final selection to true core outcomes. While there is no requirement to include outcomes that just miss the predefined cutoff for inclusion as a core outcome, after consultation with the COS Task Force, we also decided to select 80% as a threshold for noncore outcomes because this was just below the 85% threshold.

Items scored 7 to 9 by <40% of the expert panel and 1 to 3 by >15% of the panel in any round were excluded from further rounds. The Delphi process was planned to terminate either when all items were included or excluded, or when consensus no longer progressed between rounds on indeterminate items. The number of Delphi rounds varies across different COS studies. While three rounds are usually sufficient for collecting the pertinent information to reach a consensus, there is no predetermined minimum or maximum number of rounds for Delphi studies. Subsequent rounds can be added in an effort to reach consensus, but this must be balanced with the universal decrease in participation that occurs with additional rounds. To allow for an inclusive design that would maximize the number of outcomes considered, panelists were able to submit new proposed outcomes and anonymously give feedback that they felt the panelists should consider. These were added to the subsequent rounds for panel consideration.

For Round 3, participants were provided with deidentified aggregate response data from the previous round presented as a histogram bar chart. Participants' individual responses were provided only to that participant, reminding them of their outcome grading in the context of the group's aggregate rating. Intraclass correlation and current consensus COS of the group were also provided. Neither participants' names, affiliations, or individual grading was shared with any other participant. Three reminder emails were sent for each round of the Delphi process to maximize participation.

Assessment was performed using the ICC function to assess interrater test-retest reliability, in a two-way mixed-effects model, scoring for absolute agreement, with a 95% confidence interval (CI). Intraclass correlation estimates and their 95% CIs were calculated using R statistical package “ICC” version 2.3²⁰ based on a mean rating (k = 12), absolute agreement, and two-way mixed-effects model. Accepted definitions of agreement were as follows: <0.50, poor; 0.50 to 0.74, moderate; 0.75 to 0.90, good; and 0.90 to 1.00, excellent. All statistical tests were performed in the R version 4.0.2 environment.²¹

RESULTS

The Delphi process was completed in three rounds. Round 1 received free-form submission responses from 16 of 31 experts (52% response rate). There was no statistically significant difference between those invited who completed the Delphi study and those who did not based on years of experience, specialty, or geographic region. After review by the task force and elimination of duplicate items, a list of 64 unique responses was compiled for further consideration in subsequent rounds (Table 1). These were distributed to the panel for consideration.

TABLE 1 - Submissions From the Panel for Consideration as Core Outcomes

1. 6-h Mortality

2. ICU days

3. Mortality at 90 d

4. Time to death over the first 28–30 d

5. Total number of individual products within the first 24 h

6. Days out of hospital within 28–30 d

7. Composite blood usage score

8. 3-h Mortality

9. Discharged home within 28–30 d

10. Time to surgical hemostasis

11. 30-d Mortality (excluding TBI)

12. Duration of hospital stay at 30 d

13. Duration of ICU stay at 30 d

14. In-hospital mortality

15. 30-d Mortality

16. Ventilator days

17. ICU-free days

18. Morbidity (ARDS, sepsis)

19. Early mortality (3- to 6-h mortality)

20. Multiple organ failure

21. Total number of individual blood products within the first 7 d

22. Crystalloid infused volume

23. Transfusion reactions by ISBT definitions

24. Coag correction

25. 24-h Mortality

26. Time to all-cause mortality within 6 h of injury

27. TBI excluded mortality

28. Multiorgan failure before 28–30 d

29. Number of ABP units (RBCs, FP, PLTs) transfused within 24 h after arrival

30. Number of ABP units (RBCs, FP, PLTs) transfused within 7 d after arrival

31. Ventilator-free days

32. Time to OR

33. 12-h Mortality

34. Blood products received in the first 6 h

35. Blood products received in the first 24 h

36. Time of death

37. 6-h Mortality from hemorrhage

38. Wasted products

39. Number of those who achieved definitive hemorrhage control and hemostasis

40. Time to definitive hemorrhage control and hemostasis

41. Lactate

42. 8-h Mortality

43. Pulmonary complications (TRALI, need for mechanical ventilation, duration of mechanical ventilation)

44. 48-h Mortality

45. Cause of death

46. 90-d Mortality, all-cause

47. Cardiac complications (TACO)

48. 24-h Mortality from hemorrhage

49. Renal complications (AKI, need for hemodialysis)

50. Alive and at home at 28–30 d postinjury

51. Thrombotic complications

52. CAT+ in first 24 h

53. SIRS

54. Need for ICU admission

55. Need for open abdomen/damage-control surgery

56. 30-d Mortality, all-cause

57. 6 to 12 mo GOSE

58. Rescue use of hemostatic agents (e.g., rFVIIa)

59. 24-h Mortality, all-cause

60. Time to mortality

61. Return to preinjury functional level (e.g., return to work, return to school)

62. Discharge GOSE

63. Major complications (i.e., requiring pharmacological, procedural, or surgical intervention; e.g., thromboembolic events, infections)

64. 6- to 12-mo Mortality

Two proposed COSs below were added back in after Round 2 write-ins:

1. Duration of hospital stay at 30 d

2. Duration of ICU stay at 30 d

ABP, autologous blood products; AKI, acute kidney injury; ARDS, acute respiratory distress syndrome; CAT+, critical administration threshold; Coag, coagulopathy; FP, frozen plasma; GOSE, Glasgow Outcomes Score—Extended; ICU, intensive care unit; ISBT, International Society of Blood Transfusion; OR, operating room; PLT, platelet; RBC, red blood cell; rFVIIa, recombinant activated factor VII; SIRS, systemic inflammatory response syndrome; TACO, transfusion-associated circulatory overload; TBI, traumatic brain injury; TBI, traumatic brain injury; TRALI, transfusion-related acute lung injury.

In Round 2, panelists were presented with the compiled list of proposed outcomes. Of the 16 panelists completing Round 1, 13 (81%) completed Round 2. One participant responded to only 19 of 21 items in this round; these two items were evaluated using a denominator of 12 rather than 13 experts. Two items in this round met inclusion criteria as a core outcome: blood products received in the first 6 hours and 6-hour mortality. Twenty-seven items were eliminated because they had a 7 to 9 rating by <40% of the panelists. Five other items were eliminated because they were very similar to other remaining variables. This left 30 items, plus 2 items that were previously eliminated but added back as a write-in by 1 of the panelists, to be included in Round 3. We performed an ICC analysis for Round 2, with indeterminate items demonstrating an ICC of 0.74 (95% CI, 0.656–0.81) demonstrating moderate agreement.

For Round 3, 12 of the initial 16 panelists (75%) responded. The panelists were presented only with the list of 32 indeterminate items along with deidentified aggregate results and a reminder of their individual scoring from the prior round. All participants changed at least one score. In this round, 10 outcomes were eliminated as 7 of these received a 1 to 3 rating by >15% of the panelists and 3 outcomes received a 7 to 9 rating by <40%. Two additional items met the inclusion criteria: time to mortality and 24-hour mortality. Two items received a 7 to 9 rating by >80% but <85% of the panelists, so these were considered important but noncore outcomes. These were morbidity (defined as acute respiratory distress syndrome and sepsis) and early mortality (defined as mortality within 3–6 hours).

To determine the utility of proceeding to a fourth round of Delphi consensus, we subjected the scores to an ICC analysis. The indeterminate items demonstrated an ICC of 0.19 (95% CI, 0.412–0.61), indicating very poor agreement. Given this analysis, the EAST COS task force felt that further rounds of deliberation would have limited value, so the Delphi process was completed. The final list of consensus items included four core outcomes (blood products received in the first 6 hours, 6-hour mortality, time to mortality, 24-hour mortality) and two important (morbidity and early mortality) but noncore outcomes, which the panel felt warranted publication for future study consideration (Table 2).

TABLE 2 - Final Included Set of Core Outcomes and Characteristics of Value

Core Outcomes	Noncore but Important Outcome
6-h Mortality	Morbidity (defined as ARDS, sepsis)
Time to mortality	Early mortality (defined as mortality within 3–6 h)
24-h Mortality
Blood products received in the first 6 h

ARDS, acute respiratory distress syndrome.

DISCUSSION

Core outcome sets have become increasingly prominent across medicine because of the benefit of leveraging multidisciplinary expert insight while using a scientifically rigorous process. This is especially important for areas of research that are plagued with heterogeneous endpoints such as MT. Despite recent advances in our understanding of traumatic hemorrhage, existing clinical trials pertaining to resuscitation and bleeding control vary significantly in their study design and outcomes.²² In addition, many studies use traditional primary endpoints such as 24-hour and 30-day all-cause mortality; however, these are arbitrary endpoints that do not reflect the pathophysiology and timing of most hemorrhagic deaths.^22–26 Therefore, this consensus study of content experts established a COS for MT, which includes four core outcomes (6-hour mortality, time to mortality, 24-hour mortality, and blood products received in the first 6 hours) that should serve as the minimum number of outcomes reported for future studies evaluating MT. Mortality is an important outcome related to MT given the overall high rates of death in this population. This outcome is reliable, objective, and easily recorded, facilitating its use in prospective studies and retrospective studies conducted using administrative databases. It is important to note that selection of a COS does not preclude inclusion of other variables that investigators deem important when designing their trials, including physiologic variables.

While this is the first COS developed for MT using a Delphi method, our task force is not the first group to address this issue. In 2019, an international group of physicians and research scientists from the military, academia, industry, the Federal Drug Administration, and the National Heart Lung and Blood Institute convened to discuss the barriers faced by the trauma community in their efforts to improve patient outcomes.²² Their goal was to develop consensus guidelines for the design of large multicenter clinical trials, including new endpoints for clinical trials in emergency trauma research. Given that most trauma deaths due to bleeding occur within the first 3 hours after injury, and 85% of hemorrhagic deaths occur within 6 hours,^22–24 they determined that 3- to 6-hour all-cause mortality accurately captures when a hemorrhage control intervention is likely to produce a survival difference and should therefore be considered a primary outcome for randomized studies.²⁴ This decision was based off several recent trials, including the Pragmatic, Randomized Optimal Platelet and Plasma Ratios (PROPPR) trial, in which 90% of all deaths due to bleeding occurred within 6 hours of admission.²⁷

Like this other international group, through an iterative process, our panel identified three core outcomes with time intervals related to mortality, including an early (6-hour) mortality metric. The 24-hour mortality core outcome falls outside of the “early” mortality window but is nonetheless useful as a proxy for cause when cause-specific mortality cannot be assessed.²⁸ This is particularly relevant for patients in whom modern damage-control principles and MT allow temporary survival, but who ultimately die from traumatic brain injury and early multiple organ failure exacerbated by a dysfunctional immune response to trauma.^29,30

The number of blood products transfused within the first 6 hours was also found to be a core outcome in our study. This is not surprising, as it is a surrogate marker for blood loss. Results from several prospective trials have shown that earlier use of higher ratios of blood, plasma, and platelets is associated with improved survival during the first 6 hours after admission, when hemorrhagic deaths are most likely to occur.^31,32 This survival advantage has not been observed beyond 6 hours, likely because of competing nonhemorrhagic causes of death at later time points.^31,33 This also highlights the importance of stratifying by time interval and including time-dependent covariates in future trials to avoid confounding due to survival bias.³³

In addition to mortality, morbidity occurs quite frequently in the MT population with estimates ranging from 14% to 42%.^34,35 Even after hemorrhage is controlled, the cellular response of individual organ systems and the balance of inflammatory mediators over hours or days can lead to multiple organ failure weeks after the initial insult.³⁴ As such, it is not surprising that morbidity was selected as a noncore but important outcome. In fact, prospective trials on this topic already include some data on morbidity;^35–37 however, there is still heterogeneity that exists, which may benefit from even further standardization. Furthermore, early experimental treatments might also have late harmful effects; therefore, monitoring for morbidity is essential for evaluating the safety of novel hemorrhage control interventions.

The morbidity outcome was not further defined in our study, but because it was deemed an important but noncore outcome, it would be up to future investigators whether to include and further define this outcome in any future clinical trials. While other measures related to morbidity and cause of death are important in trials evaluating massive transfusion, those specific variables would be dictated by the aims and endpoints of the particular study and may not merit inclusion in all studies on MT. In addition, nonhemorrhagic deaths occur later during hospitalization. The core outcomes identified in this Delphi pertain to early mortality and have been identified as critical to report in future research on this topic.

The other noncore outcome selected in our study was early mortality defined as 3 to 6 hours. We find it interesting that 3- to 6-hour mortality was selected as a noncore outcome despite it being encompassed by the 6-hour mortality core outcome. We can only speculate that the experts felt that “early mortality” was better defined as within 6 hours of injury and should therefore be a core outcome. However, some experts felt that having a midrange outcome of 3 to 6 hours would also be beneficial to see if the 0- to 3-hour versus the 3- to 6-hour period is the main driver of mortality at 6 hours. These findings help support the use of Delphi to clarify such ambiguities in research and scientific publishing.

There are some limitations to consider with this study. We attempted to achieve interrater consensus with only three rounds of the Delphi; however, the ICC of 0.19 indicates very poor reliability, and further deliberation was unlikely to move the panel from poor agreement to supermajority agreement in the absence of new information. Had the study been stopped at Round 2, it would not have been a true Delphi study, as the feedback provided between Rounds 2 and 3 is a key feature of this type of consensus method. Subsequent rounds after Round 3 could have led to a further decline in participation and unnecessary prolongation, which is a well-recognized limitation of the Delphi consensus method. Also, despite our efforts, this Delphi was not able to recruit patients into the panel and therefore lacks insight into patient values and patient-centered outcomes of interest. Future work to develop trauma-related patient support networks like those for patients with medical diseases³⁸ may help mitigate this limitation if future revision of this COS is undertaken. There was a lack of pediatric surgery–trained trauma surgeons on the panel, although many of the panelists care for pediatric trauma patients. While there are certainly other important outcomes in trials evaluating MT, to adhere to the results of the Delphi consensus, we were unable to add any outcomes that were not endorsed by the panel of experts. Finally, it is important to note that COSs are currently designed specifically for research, not clinical practice. However, there may eventually be a role for the inclusion of high-quality COS in national quality improvement databases.

Despite these limitations, our study has many strengths such as the clear a priori consensus definitions used, the multidisciplinary international panel who participated, and the strict adherence to the Delphi principles. While our Delphi method for achieving consensus differs from the in-person approach, the two techniques are complementary. While the latter method is likely to encourage more discussion, it is also susceptible to certain flaws such as a strong personality or someone with higher academic stature dominating the process and unduly influencing others' opinions. The Delphi approach, on the other hand, maintains the anonymity of respondents by using an electronic exchange of information in a controlled feedback process.³⁹

It is important to note that another challenge with developing a COS is that new information pertaining to the topic of interest may become available over time. Therefore, a COS must be reassessed and updated to reflect this new information. It must again be emphasized that the outcomes within this COS serve as the bare minimum of research outcomes, and future MT researchers are encouraged to include additional noncore outcomes as necessary for their intended study hypothesis. In addition, a COS Delphi survey does not rank core outcomes once they reach consensus, nor does it require that particular outcomes be designated primary or secondary outcomes. Because no weight or rank is assigned to the selected core outcomes, researchers may choose their primary outcome depending on what is being studied.

CONCLUSION

This COS for MT identified four core outcomes (6-hour mortality, time to mortality, 24-hour mortality, and blood products received in the first 6 hours), which represent a minimum set of outcomes that should be reported in future MT studies. Use of these core outcomes will help standardize and optimize data collection and facilitate the capability for pooled research.