Hemorrhage and subsequent cardiovascular collapse (shock) represent primary mechanisms in both battlefield and civilian trauma deaths (1–3). It is estimated that approximately 25% of deaths associated with bleeding on the battlefield are potentially preventable with adequate detection, intervention, and subsequent treatment of the proximate cause of the hemorrhage (3). Early detection and timely intervention in patients with blood loss are confounded, however, by a variety of physiological compensatory mechanisms (1, 4, 5). These mechanisms can maintain heart rate (HR), blood pressure, and other standard vital signs in or near the normal clinical range during loss of up to 30% of blood volume (4, 6). Without early detection and intervention, greater levels of compensation can be followed by rapid and precipitous “cardiovascular collapse,” which is marked by a sudden and dramatic fall in blood pressure and is sometimes the first clear clinical sign of life-threatening hemorrhage (6, 7). Importantly, these compensatory mechanisms limit the ability of care providers to detect the risk of imminent cardiovascular collapse using traditional vital signs. As such, the need for a composite measurement that indicates the physiological reserve to compensate for relative blood volume deficits is evident. We have described this physiological measurement as the “compensatory reserve” (8). In this context, emerging evidence indicates that progressive central volume loss leads to measurable changes in the features of arterial waveforms (4, 5, 8–16). This information should be leveraged to provide real-time decision support, guide timely intervention, monitor resuscitation effectiveness, and reduce the risk of catastrophic cardiovascular collapse.
The Compensatory Reserve Index (CRI) represents a new paradigm for measuring the integrated cardiopulmonary mechanisms (e.g., tachycardia, vasoconstriction, breathing) that compensate for relative blood volume deficit (5, 8, 11, 12, 14). Measurement of the CRI is based on application of feature extraction techniques with machine learning to reveal subtle changes in waveform features that are associated with declining central blood volume. As such, the CRI represents all factors that contribute to compensatory mechanisms including baroreflexes and metaboreflexes, various muscle contractions, respirations, and so on. The CRI was designed to prospectively identify ongoing loss of central blood volume and subsequently estimate the progression toward hemodynamic decompensation (onset of shock) in a bleeding patient, well in advance of changes in standard vital signs (8).
The CRI has been shown to correlate with central hypovolemia in human subjects under laboratory conditions (8, 11, 17, 18) and controlled low-volume blood loss (<500 mL) (12, 14); however, the utility of the CRI to track moderate volume blood loss has not been tested. In the present experiment, we conducted a controlled hemorrhage of greater than 20% of blood volume in humans to test the hypothesis that the CRI would provide greater sensitivity and specificity to detect moderate-volume blood loss compared with traditional vital signs.
Subjects and ethical approval
Twenty-five healthy nonsmoking men and women between the ages of 18 and 55 years volunteered to participate in this study. All experimental procedures were conducted in accordance with a protocol reviewed and approved by the Duke University institutional review board and the institutional review board of the Office of Human Research Protection under the US Army Medical Research and Materiel Command. All subjects underwent a medical history and physical examination by a physician to ensure that they had no previous or current medical conditions that might preclude their participation. Standard preoperative screening tools (e.g., American Heart Association/American College of Cardiology assessment of cardiovascular risk) were used to ensure that subjects had no chronic systemic diseases. Subjects were screened for a history of cardiovascular disease, hypertension, diabetes, or any other diseases/conditions that might negatively influence their ability to tolerate moderate but temporary blood loss. Because selection of subjects focused on a population relevant to the military and high body mass index is associated with attenuation of compensatory responses related to baroreflex sensitivity (19), obese subjects (body mass index >30 kg/m2) were not enrolled. Subjects on prescription drugs of any type (except oral contraceptives) were not studied. Female subjects were not pregnant, as confirmed by a urine pregnancy test 1 h prior to experimentation. Subjects were instructed to maintain their normal sleep patterns, refrain from exercise, and abstain from caffeine and other autonomic stimulants including nonprescription drugs for at least 24 h prior to each experiment. Subjects received a verbal and written briefing of all procedures and risks associated with the study and were made familiar with the laboratory, the protocol, and the instrumentation. Subjects were encouraged to ask questions of the investigators before giving their written informed consent to participate.
Subjects were instrumented with a standard 3-lead electrocardiogram recording for determination of HR, and an integrated photoplethysmograph (PPG)/volume-clamp pressure sensor (ccNexfin Blood Pressure Monitor; Edwards Lifesciences, Irvine, Calif) was placed on the middle finger of the right hand at heart level to record beat-by-beat arterial pressure. The ccNexfin was used to continuously calculate stroke volume and blood pressure for the duration of the experiment. Arterial oxygen saturation (SpO2) was measured with a standard fingertip pulse oximeter (Nonin 9560; Nonin, Minneapolis, Minn). Compensatory Reserve Index was estimated using the 9560 PPG waveform.
A large-bore peripheral i.v. line was inserted into an antecubital vein using aseptic technique. This catheter allowed blood withdrawal and was available in case the blood loss caused hypotension (>30% reduction in blood pressure from baseline) that required intervention to support the subjects’ blood pressure.
Progressive stepwise reductions in central blood volume were induced by a graded blood loss protocol of 20% estimated blood volume removed in aliquots up to ∼333 mL each. A 3-min “plateau” period was used for data collection following each aliquot of blood withdrawal. Total blood volume was estimated for men at 75 mL/kg and for women at 65 mL/kg. As a safety precaution, the total maximum amount of blood removed was limited to 4 U (∼1,333 mL) for men and 3 U of blood (1,000 mL) for women. Fifteen to 20 min was required for each stage of blood withdrawal to occur via the venous catheter. As such, the total time of controlled hemorrhage was approximately 60 (women) to 80 (men) min. Withdrawn blood was stored using standard preservative/anticlotting solutions used in perioperative hemodilution and cell salvage protocols. At the end of the blood loss protocol, the collected blood was reinfused into the subjects.
A machine-learning framework for estimating compensatory reserve
As detailed previously (8, 11), state-of-the-art feature extraction and machine-learning techniques were used to construct a model for estimating CRI based on Nonin PPG recordings obtained from humans during progressive central hypovolemia (8). The resulting algorithm estimates the proportion of physiological reserve available to compensate for changes in effective circulating blood volume. It does so by normalizing and comparing waveform features within a sliding 30-heartbeat window with those in the model. This comparison occurs at each heartbeat to produce beat-to-beat CRI values, which can be trended over time. For clinical simplicity, the CRI has been normalized on a scale of 1 to 0 (100%–0%), where 1 reflects the maximum capacity of physiological mechanisms (e.g., baroreflexes, respiration) to compensate for reduced central blood volume, and 0 implies imminent cardiovascular instability and decompensation. Values between 1 and 0 indicate the proportion of compensatory reserve available to compensate for further volume loss.
Compensatory Reserve Index data were generated at each heartbeat and averaged to provide one value for each 3-min plateau period. The Pearson product correlation coefficient was used to assess the relationship between changes in intravascular blood volume and changes in blood pressures, HR, oxygen saturation, stroke volume, cardiac output, systemic vascular resistance, and the CRI. The probabilities of observing chance effects that changes in the dependent variables over changing circulating blood volumes (time) were different from “zero” change are presented as exact P values obtained from repeated-measures analyses of variance. The effectiveness of each of the binary classifiers (one per parameter) was assessed using receiver operating characteristic area under the curve (ROC AUC) analysis and estimation of sensitivity and specificity. All data are expressed as means ± 95% confidence interval (95% CI).
A total of 25 patients were recruited and participated as subjects. Data were excluded from five subjects because of corrupted electronic data files (n = 2) and development of symptoms prior to completion of blood withdrawal (n = 3). Thus, data from 20 subjects were available for analysis.
Blood pressures (systolic [SBP], diastolic [DBP], mean [MAP]) and SpO2 were unchanged during progressive reductions in blood volume, whereas HR increased during the last two stages of blood withdrawal (Fig. 1). Although changes in SBP (P ≤ 0.0014) and HR (P ≤ 0.0132) across volume changes were different from zero change, maximum average values remained within clinical norms (119 mmHg and 75 beats/min; Table 1). On the other hand, a linear reduction with slope greater than 0 (Table 1) in CRI was observed with progressive loss of blood volume (Fig. 1).
Progressive blood loss resulted in average reductions of 15.7% in stroke volume (P < 0.0001) and 33% in CRI (P < 0.0001) and an average increase in HR of 15.6% (P ≤ 0.0132), whereas changes in cardiac output (P ≥ 0.8141) and peripheral vascular resistance (P ≥ 0.9997) were not statistically distinguishable from zero change. Linear correlation coefficients (r) between blood loss volume and stroke volume, HR, and CRI were 0.947, −0.789, and 0.960, respectively, but statistically less (range, −0.547 to 0.428) for all other hemodynamic variables (Table 1). Likewise, the ROC AUC (0.901) for CRI demonstrated statistically greater specificity (0.759) and sensitivity (0.800) compared with all other hemodynamic variables (Table 1).
Figure 2 shows the individual scatter plots of 20 subjects comparing stepwise blood volume loss (closed circles) and blood volume restoration (open circle) to compensatory reserve as measured by the CRI algorithm. Individual correlation coefficients ranged from 0.907 to 0.988, with a group average of 0.960 ± 0.019. Following complete whole-blood reinfusion, the group average for the CRI value returned to 0.90 ± 0.02 compared with the baseline value of 0.92 ± 0.02 prior to blood withdrawal.
In the present investigation, human subjects were exposed to a progressive controlled hemorrhage that resulted in an average maximal reduction of circulating blood volume of approximately 1.2 L. We hypothesized that a model developed with the use of state-of-the-art feature-extraction and machine-learning techniques would provide earlier and more accurate estimates of blood volume loss in individual subjects with varying compensatory responses. To test this hypothesis, we compared standard vital sign responses (arterial blood pressures, HR, SpO2) with compensatory reserve measurements obtained from noninvasive arterial pulse waveform features. Our hypothesis was supported as shown in Figure 1. Consistent with our previous observations (4, 5, 8), data from this study reaffirmed that arterial blood pressures and oxygen saturation were essentially unaltered during progressive blood loss, whereas HR increased only slightly in the latter phases of hemorrhage. In contrast, the compensatory reserve as measured by CRI was progressively reduced in a manner that significantly correlated with the loss of circulating blood volume. Furthermore, sensitivity and specificity of the standard vital signs were only moderate as indicated by the ROC AUC ranging from 0.51 to 0.67, compared with an AUC = 0.90 for the CRI (Table 1). As such, measuring the magnitude of the reserve to compensate for blood loss proved to be the most robust marker of individual variation between subjects (Fig. 2) because it is a more sensitive and specific reflection of the integrated capacity of regulatory mechanisms to maintain adequate perfusion and oxygen delivery to the tissue.
Several unique features of measuring the compensatory reserve are demonstrated by the data presented in Figure 2. First, the significant correlation of reduced compensatory reserve with the volume of blood loss across every subject indicates the ability of the model to provide individualized assessment of compensatory status. Second, the differences in slopes of the stimulus-response (blood volume–CRI) relationship represent the individual reserve required to compensate for the reduction in circulating blood volume. Previous experiments using lower-body negative pressure as a model for inducing progressive hemorrhage-like decompensation (8, 17) support the notion that those subjects who are at greatest risk of early decompensation (e.g., shock) can be identified by a steeper (faster) reduction in compensatory reserve (e.g., subjects 1 and 3) compared with those who require less depletion of their reserve to maintain stable vital signs (e.g., subjects 2 and 6). Third, the complete restoration of the group average compensatory reserve following replacement of circulating volume (Fig. 2, open circles) suggests that the algorithm could provide a monitoring capability for guiding controlled resuscitation strategies. Finally, it can be noted that no subject began the experiment with a CRI = 1.0. The algorithm developed to calculate CRI was based on a lower-body negative pressure protocol in which the initial baseline stage was executed under controlled experimental conditions (e.g., adequate hydration, sleep, food intake, etc.) with subjects in the supine posture (i.e., optimum central blood volume). Such experimental control was not possible for the volunteers in the present experiment who were placed in a seated position for blood withdrawal. As such, it was not surprising that the average CRI in the present study was slightly below 1.0 at 0.90.
In two previous investigations (12, 14), the average compensatory reserve was measured before and after low volume blood loss induced by a standard donation of ∼450 mL whole blood. These independent studies produced nearly identical results with an average 16% to 19% change in CRI from 0.74 to 0.62 (12) and 0.78 to 0.63 (14). The present study extends those initial studies by increasing blood loss by a magnitude of more than 2.5-fold. We subsequently expected that the CRI would fall to a level significantly lower than 0.60. Against expectations, we observed that CRI fell to an average of only 0.59 (Table 1). There are at least two possible explanations for the relatively small difference in CRI after blood donation in the current compared with the former studies based on the data. First, the initial baseline CRI of the subjects in the present investigation was significantly higher (0.90) than the average baseline CRI of the subjects in the former two studies (0.74–0.78). As such, the subjects of the present study relied on a 2-fold greater absolute (0.30 vs. 0.15) and relative (33% vs. 18%) amount of their reserve to compensate for the 2.5-fold larger blood loss. This comparison underscores the importance of maintaining a “full tank” of reserve capacity (e.g., maximal hydration state) in order to provide the optimum capacity for compensation. Second, the slower rate of hemorrhage during controlled hemorrhage in the present study may have influenced the rate of utilization of the compensatory reserve. Consistent with this notion is our previous observation that a faster rate of fall in CRI during progressive reduction in central blood volume is associated with earlier development of hypotension (17). The average rate of blood loss during the initial studies (12, 14) was ∼50 mL/min (∼450 mL withdrawn over a mean duration of 9 min) in contrast to a slower rate of ∼20 mL/min (∼1,000-mL average blood withdrawal over a mean duration of 53 min) in the present investigation. As such, the similar remaining reserve for compensation (0.59) to 1.2 L of blood loss compared with the remaining compensatory reserve (CRI = 0.62–0.64) with 450 mL blood loss is likely due to the ability of compensatory mechanisms to better accommodate a slower rate of blood loss.
The data of three subjects were not included in the final analysis because they failed to complete their goal-directed blood loss protocol due to symptoms of presyncope. In these three cases, the CRI values fell to 0.18, 0.14, and 0.11, respectively (i.e., subjects depleted >80% of their compensatory reserve), more than twice the average amount of compensatory reserve used by the 20 subjects (38%) who completed the blood loss protocol with an average CRI of 0.62. These results are consistent with our previously reported findings that hemodynamic decompensation occurs at an average CRI of 0.10 to 0.13 in subjects exposed to progressive reductions in central blood volume (8) and substantiate the high specificity as well as sensitivity of the CRI as an early indicator of progressive blood loss.
Measurement of stroke volume has proven to provide useful information about circulating volume status because of the relationship between cardiac filling and central blood volume. Because stroke volume is associated with features of the arterial pulse waveform (9, 10, 20), it is not surprising that stroke volume showed a high correlation with blood loss similar to that of CRI (Table 1). However, the inability of changes in stroke volume to differentiate individuals with high compared with low tolerance to reduced central blood volume (8, 13) is reflected by the lower ROC AUC and specificity for stroke volume compared with the CRI in the present investigation (Table 1). As such, our findings support the notion that, compared with measurements of stroke volume, advanced machine-learning techniques designed to identify real-time subtle changes in the patterns and features of arterial waveforms can improve the fidelity of decision support related to diagnosis and care of patients with significant blood loss in an emergency medical setting when standard vital signs remain within normal clinical levels.
Individual measurements that represent mechanisms of compensation have been examined as potential early markers of imminent cardiovascular instability. However, significant interindividual and intraindividual variance has resulted in failure to provide a clinical tool from measures of vital signs with acceptable specificity (8, 12, 14, 21–23). Traditional approaches are limited by algorithms based on multivariate regression models that represent static average responses of patient populations with time delays, and can require demographic information such as age, gender, height and weight (24). The algorithm for determining CRI is unique in that it captures the status of individual patients in real time by estimating the amount of reserve remaining for compensation during blood loss without having to ’know’ demographics or any other information. Although other recent studies have analyzed PPG waveforms in an effort to determine patient status, but have demonstrated larger than desired differences in predictive ability between algorithm development and validation samples, as measured by ROCAUC values (25). On the other hand, the CRI has demonstrated no difference in AUC when comparing the results of this study to previously published results from a different patient sample, which suggests greater reliability in predictions (14). This detailed model that captures individual status is best demonstrated by the different slopes of regression for individual subjects (Fig. 2).
Like all investigations, our study is not without limitations. Although the present study was purposely designed with the focus on a population relevant to the military, we cannot yet extrapolate the accuracy of the CRI to elderly patient populations with comorbidities such as heart or vascular diseases. Likewise, stepwise reductions in blood volume do not mimic the more clinically relevant scenario of continuous bleeding. However, our current laboratory LBNP protocols using continuous ramp profiles (5) and clinical data collection from patients with hemorrhage associated with penetrating trauma, dengue hemorrhagic fever, renal dialysis, cardiac surgery, and child birth demonstrate the ability of the technology to track continuous changes in circulating blood volume (unpublished data). We hypothesize that the fundamental basis of measuring features of the arterial waveform should be applicable in all populations and clinical conditions because the total integration of all central (cardiac) and peripheral (vascular resistance) compensatory mechanisms should be reflected in changes of specific features in either the ejection wave or reflective wave. Data to test this hypothesis must await further experimentation that will include patients with trauma, older subject populations with and without comorbidities, as well as varying rates of continuous blood loss.
Measurement of the compensatory reserve is based on a novel mathematical model capable of identifying the specific status of individuals relative to a deficit in circulating blood volume well in advance of clinically significant changes in current vital signs. Based on the fundamental biophysics of pressure-flow relationships within the cardiovascular system, this model analyzes and compares the entirety of arterial waveform features that reflect the status of compensation reserve for a particular central blood volume. The results of the present investigation demonstrate the physiological relationship between significant progressive blood loss and the underlying absolute reserve capacity that dictates differences among individuals.
Our current capabilities for assessing the status of a bleeding combat casualty are designed to provide medics with raw population-driven vital sign information rather than to generate statistically unbiased, continuous “interpreted” information required to direct early intervention. In the present investigation, we challenged traditional bias by introducing the continuous noninvasive measurements of the features of peripheral arterial waveforms obtained from finger PPG and introduced these signals into a machine-learning model designed to measure the compensatory reserve with calculation of the CRI. This new paradigm can recognize any compromise to the compensatory reserve well before an individual experiences signs and symptoms of cardiovascular instability. As such, the CRI can provide the combat medic the capability to foresee progression toward hemodynamic instability and thereby promote earlier intervention when the physiology is less complex and more likely to respond to therapy. The CRI algorithm can be integrated into any standard monitor that generates an arterial waveform, including the finger pulse oximeter available in the medical kits of US Army combat medics and civilian first responders. Such a capability should better inform triage decisions in the prehospital setting and may prove to be a “game changer” for improved patient outcomes.
The authors thank the Duke Human Pharmacology and Physiology staff who contributed to the clinical care of the research subjects: Maria A Santoro and CRC and CRNA staff Amelia T Fiore, Kathryn E Gosnell, Derrick T King, Dayna Seguin, Matthew J Stamper, Robert G. Taylor, Jr, and Sara Zucco.
1. Bellamy RF: The causes of death in conventional land warfare: implications for combat casualty care research. Mil Med
149: 55–62, 1984.
2. Eastridge BJ, Hardin M, Cantrell J, Oetjen-Gerdes L, Zubko T, Mallak C, Wade CE, Simmons J, Mace J, Mabry R, et al.: Died of wounds on the battlefield: casuation and implications for improving combat casualty care. J Trauma
71 (1): S4–S8, 2011.
3. Eastridge BJ, Mabry RL, Seguin P, Cantrell J, Tops T, Uribe P, Mallett O, Zubko T, Oetjen-Gerdes L, Rasmussen TE, et al.: Death on the battlefield (2001–2011): implications for the future of combat casualty care. J Trauma Acute Care Surg
73 (6 Suppl 5): S431–S437, 2012.
4. Convertino VA, Moulton SL, Grudic GZ, Rickards CA, Hinojosa-Laborde C, Gerhardt RT, Blackbourne LH, Ryan KL: Use of advanced machine-learning techniques for noninvasive monitoring of hemorrhage. J Trauma
71: S25–S32, 2011.
5. Convertino VA: Blood pressure measurement for accurate assessment of patient status in emergency medical settings. Aviat Space Environ Med
83: 614–619, 2012.
6. Cottingham A: Resuscitation of traumatic shock: a hemodynamic review. AACN Adv Crit Care
17 (3): 317–326, 2006.
7. Convertino VA, Ludwig DA, Cooke WH: Stroke volume and sympathetic responses to lower-body negative pressure reveal new insight into circulatory shock in humans. Auton Neurosci
111: 127–134, 2004.
8. Convertino VA, Grudic GZ, Mulligan J, Moulton SL: Estimation of individual-specific progression to impending cardiovascular instability using arterial waveforms. J Appl Physiol
115: 1196–1202, 2013.
9. Convertino VA, Cooke WH, Holcomb JB: Arterial pulse pressure and its association with reduced stroke volume during progressive central hypovolemia
. J Trauma
61: 629–634, 2006.
10. McGrath SP, Ryan KL, Wendelken SM, Rickards CA, Convertino VA: Pulse oximeter plethysmographic waveform changes in awake, spontaneously breathing, hypovolemic volunteers. Anesth Analg
112 (2): 368–374, 2011.
11. Moulton SL, Mulligan J, Grudic GZ, Convertino VA: Running on empty? The Compensatory Reserve Index. J Trauma Acute Care Surg
75: 1053–1059, 2013.
12. Nadler R, Convertino VA, Gendler S, Lending G, Lipsky AM, Cardin S, Lowenthal A, Glassberg E: The value of non-invasive mesurement of the Compensatory Reserve Index in monitoring and triage of patients experiencing minimal blood loss. Shock
42: 93–98, 2014.
13. Rickards CA, Ryan KL, Cooke WH, Convertino VA: Tolerance to central hypovolemia
: the influence of oscillations in arterial pressure and cerebral blood velocity. J Appl Physiol
111: 1048–1058, 2011.
14. Stewart C, Mulligan J, Grudic GZ, Convertino VA, Moulton SL: Detection of low volume blood loss: the Compensatory Reserve Index vs. traditional vital signs. J Trauma Acute Care Surg
77: 892–898, 2014.
15. Xu D, Ryan KL, Rickards CA, Zhang G, Convertino VA, Mukkamala R: Improved pulse transit time estimation by system identification analysis of proximal and distal arterial waveforms. Am J Physiol Heart Circ Physiol
301: H1389–H1395, 2011.
16. Zhang G, Ryan KL, Rickards CA, Convertino VA, Mukkamala R: Early detection of hemorrhage via central pulse pressure derived from a non-invasive peripheral arterial blood pressure waveform. Conf Proc IEEE Eng Med Biol Soc
17. Poh Y, Carter R III, Hinojosa-Laborde C, Mulligan J, Grudic GZ, Convertino VA: Respiratory pump contributes to increased physiological reserve for compensation duirng simulated haemorrhage. Exp Physiol
99 (10): 1421–1426, 2014.
18. Van Sickle C, Schafer K, Mulligan J, Grudic GZ, Moulton SL, Convertino VA: A sensitive shock index for real-time patient assessment during simulated hemorrhage. Aviat Space Environ Med
84 (9): 907–912, 2013.
19. Beseke SD, Alvarez GE, Ballard TP, Davy KP: Reduced cardiovagal baroreflex gain in viseral obesity: implications for the metabolic syndrome. Am J Physiol Heart Circ Physiol
282: H630–H635, 2001.
20. Reisner AT, Xu D, Ryan KL, Convertino VA, Rickards CA, Mukkamala R: Monitoring non-invasive cardiac output and stroke volume during experimental human hypovolaemia and resuscitation. Br J Anaesth
106 (1): 23–30, 2011.
21. Hinojosa-Laborde C, Rickards CA, Ryan KL, Convertino VA: Heart rate variability during simulated hemorrhage with lower body negative pressure in high and low tolerant subjects. Front Physiol
2: 85, 2011.
22. Rickards CA, Ryan KL, Ludwig DA, Convertino VA: Is heart period variablity associated with the administration of lifesaving interventions in individual prehospital trauma patients with normal standard vital signs? Crit Care Med
38: 1666–1673, 2010
23. Ryan KL, Rickards CA, Ludwig DA, Convertino VA: Tracking central hypovolemia
with ECG in humans: cautions for the use of heart period variability in patient monitoring. Shock
33 (6): 583–589, 2010.
24. Mackenzie CF, Wang Y, Hu PF, Chen SY, Chen H, Hagegeorge G, Stansbury L, Shackelford S: Automated prediction of early blood transfusion and mortality in trauma patients. J Trauma Acute Care Surg
76 (6): 1379–1385, 2014.
25. Mackenzie CF, Gao C, Hu PF, Anazodo A, Chen H, Dinardo T, Imle PC, Hartsky L, Stephens C, Menaker J, Fouche Y, Murdock K, Galvagno S, Alcorta R, Shackelford Sand the ONPOINT Study Group: Comparison of decision-assist and clinical judgment of experts for prediction of life saving interventions. Shock
43 (3): 238–243, 2015.