Methodologic Quality of Randomized Controlled Clinical Trials in Sepsis
Problems with Using Mortality as an Outcome Measure.
The methodology of randomized clinical trials in sepsis using mortality as the primary end point has improved over time (1–3). However, the methodology of studies using surrogate outcomes remains inadequate. In fact, many of these studies do not report explicit outcomes or end points. What constitutes sufficient outcome reporting in sepsis trials continues to be controversial. Most agree that it would include short-term (28-day) all-cause mortality along with secondary end points including intensive care unit (ICU) and hospital lengths of stay, organ dysfunction, renal replacement therapy, physiologic variables, and complications such as nosocomial infection (3). Some argue that the only real outcomes of importance in intensive care are severity of illness–adjusted mortality, residual morbidity, and cost (4). Unfortunately, the traditional fixed end point of short-term mortality is not practical in pediatric sepsis trials because of the relatively low mortality rate in pediatric sepsis, and thus far, morbidity and cost can be difficult and controversial to measure (5).
Many investigators continue to argue that the primary outcome measure for sepsis trials generally should be mortality; however, multiple problems exist with the 28-day all-cause mortality end point, particularly for pediatric trials:
- 1) A large number of subjects are required to demonstrate benefit, particularly as the risk of mortality continues to decrease.
- 2) The choice of 28 days is arbitrary and may be too short to reflect the effect of multiple organ dysfunction syndrome on mortality.
- 3) Attributing death to sepsis as opposed to underlying disease is difficult, and most children with sepsis have underlying diseases that are unique from adults.
- 4) Withdrawal-of-care issues cloud mortality as an end point. This question is critical because most death in pediatric ICUs involves withdrawal of care.
- 5) Mortality is insensitive to other important clinical outcomes. For example, mortality does not account for subjects with significant neurologic morbidity after a sepsis event.
An illustrative power/sample size calculation considering mortality as an end point for a trial in severe pediatric sepsis is instructive: to achieve a mortality reduction in severe pediatric sepsis from 24% to 20% (relative reduction of 16.7%, roughly equal to that seen by Annane et al. (6)) would require >3,200 total subjects to demonstrate this degree of mortality reduction with an alpha of .05 and power of .80 (7).
Alternative end points will increasingly become important as mortality from severe sepsis continues to decrease, and under appropriate circumstances, major morbidities could also be considered as primary end points. One measure of morbidity is organ dysfunction, and validated organ dysfunction scores for children have been developed (8, 9). Patient-centered outcomes are also extremely important, including quality and duration of life, quality of dying, and the effect of a patient’s health on loved ones (5). Finally, the cost of medical care must also be considered.
Characteristics of Good Surrogate Markers.
A laboratory measurement or physical sign may be used as a substitute for clinically meaningful end points that directly assess how a patient feels, functions, or survives. Therapeutic-induced changes on a surrogate end point are expected to reflect important changes in a clinically meaningful end point (10). Surrogate markers must be known or highly suspected to be in the causal pathway of the related patient-centered outcome (11). By definition, surrogate outcomes are very sensitive to treatment effects (and easier to measure) and therefore are considered to be more responsive than usual patient-centered outcomes (5). For example, a biomarker may occur at some threshold level much sooner than a corresponding related clinical symptom. Not only would this marker be more sensitive, but it may permit earlier intervention, presumably with a therapy that would affect the clinical sign by virtue of its effect on the biomarker.
What is really at issue concerning surrogate markers is the definition of clinical benefit and how to measure it (12). That is, there may be an important disconnect between the biological effect (does it work?) and an actual clinical benefit (does it help?). For example, in the recent trial of the nitric oxide synthase inhibitor, NG-methyl-L-arginine hydrochloride (546C88), in adults with severe sepsis, the intervention promoted resolution of shock but actually seemed to increase mortality (13, 14).
During the development of sepsis, staging systems offer potential alternative outcomes variables for clinical trials. Staging itself can be associated with outcome, although staging, like severity of illness, is usually not considered an outcome. For example, the PIRO staging system assesses a patient’s predisposition, the type of insult, the patient’s clinical response, and organ dysfunction, which generate potential outcome variables involving the microbe and associated toxins, clinical and genetic makeup of the host, and number and intensity of organ dysfunctions, respectively (15). Assessing measures, markers, and mediator surrogate end points in clinical sepsis may identify processes that mediate the disease state of interest at a time when an intervention can reasonably alter disease progression. A modification of Koch’s postulates for identifying the role of mediators in sepsis has been proposed (15):
- 1) The mediator is present in all patients having the disease.
- 2) Administration of the mediator to an experimental animal or human reproduces the clinical features of the disease.
- 3) Neutralization of the mediator before experimental induction of the disease should prevent the development of the disease.
- 4) Neutralization of the mediator after the experimental induction of the disease must attenuate its subsequent severity.
Importance of illness severity and natural history of sepsis is also key in terms of choosing appropriate outcome measures. For example, in the pediatric trial examining bactericidal-permeability increasing protein, the fulminant progression of meningococcal sepsis and the timing of intervention in relation to the clinical explosion of the inflammatory cascade likely contributed to the inability to demonstrate a beneficial effect in reducing mortality (16, 17). Investigators involved in this trial emphasized the need to develop end points other than mortality; for example, death vs. survival with severe/moderate morbidity vs. survival with no or mild morbidity (17). In designing new trials using surrogate end points, it will be important to establish evidence of infection, biological plausibility of the intervention target, and illness severity (8).
Although a mortality signal in pediatric sepsis trials is unlikely to occur, devastating morbidity events should be considered as valid outcome measures (18). These events include moderate or severe amputation, deterioration of the Pediatric Overall Performance Category score of two or more scales, and ongoing neurologic or other organ dysfunction at day 28.
Organ Dysfunction Resolution.
As sepsis therapy is designed to support failing organ systems, it is rational to consider rate of resolution of organ system dysfunction as an important, morbidity-related, alternate outcome. Essentially, three methods may be utilized for quantification of organ dysfunction in a sepsis trial, namely, time to recovery of organ failure, new organ failures, or organ-failure–free days (19). As a number of pediatric studies have clearly demonstrated that the risk of mortality is related to the number of organ dysfunctions (20–24), this alternate outcome measure seems rational. Utilizing the Sequential Organ Failure Assessment score, it was demonstrated that activated protein C (drotrecogin alfa) was associated with faster resolution of cardiovascular (p = .009) and respiratory (p = .009) dysfunctions and slower onset of hematologic (p = .041) organ dysfunction (25).
Multiple investigators have used various scoring systems to define and quantitate organ dysfunction (20–24), including those of an International Pediatric Severe Sepsis Consensus Conference convened in 2002 (26). The only system that has been validated is the Pediatric Logistic Organ Dysfunction score (24). This score considers neurologic, cardiovascular, renal, pulmonary, hematologic, and hepatic organ dysfunctions and is highly correlated with mortality. To date, no investigation has ascertained that a therapeutic intervention can result in faster normalization of Pediatric Logistic Organ Dysfunction scores in septic children. At the current time, Eli Lilly and Company is conducting the pediatric investigation of activated protein C (F1K-MC-EVBP, phase III study) with the primary objective being time to complete resolution of a composite of cardiovascular, pulmonary, and renal organ dysfunctions.
In practical terms, organ-failure–free days are enumerated as the number of days after enrollment to day 28 that a patient is alive and free from clinically significant organ failure as defined by the Pediatric Logistic Organ Dysfunction parameters (24). This outcome reflects differences in mortality, organ failure days among survivors, or both, and it assumes that any treatment that decreases the duration of organ failure among the survivor also increases the number of survivors and that a decrease in organ failure represents a decreased morbidity burden. Accordingly, the number of days from enrollment to day 28 in which a patient does not experience organ dysfunction is recorded, and on day 28, survivors are assigned a score corresponding to the number of days they did not experience organ dysfunction. Nonsurvivors are assigned zero organ-failure–free days or a score corresponding to the number of days they did not experience organ dysfunction. This number is subtracted from the lesser of 28 or the number of days to death. The following summarizes one way to calculate organ-failure–free days, but it is not the only way (e.g., there are multiple methods of imputation for missing values). The most abnormal daily value for each variable is used for scoring. If a particular test was not done, previous data are carried forward. If appropriate data were never obtained, the value is assumed to be normal, but if a particular data point is missing, it is assumed to be the worst possible variation. Organ-failure–free days need not be contiguous. When a patient is discharged from the pediatric ICU to home or a rehabilitation facility before day 28, the patient is considered to be organ failure free. If a patient is transferred to another ICU before day 28, it is assumed that on the days for which there is no information available, the subject was stable from the last measurement. Because organ-failure–free days are not normally distributed, data are usually represented as the median number of days to day 28 that patients are free from all organ failures, and these data are analyzed utilizing a Wilcoxon’s test, which weights mortality and organ failure equally. In terms of pediatric clinical trials, organ-failure–free days is being used in the pediatric activated protein C trial noted above and has also been used as a clinical end point for the pediatric prone-positioning study (27) and in a recent trial examining the potential benefit of exogenous pulmonary surfactant in children with acute lung injury (D. F. Wilson, personal communication) (28).
Other end points that reflect morbidity may also be expressed in terms of “free” days. Such end points include ventilator-free days, shock-free days, complication-free days, vasoactive-drug–free days, dialysis-free days, and ICU-free days (7, 12). Of these, ventilator-free days has received the most prominence and has been utilized by the ARDS Network as an efficacy outcome measure in clinical trials of treatments for acute respiratory distress syndrome (29). Ventilator-free days are defined as the number of days between successful weaning from mechanical ventilation to day 28 after study enrollment. It reflects differences in mortality, ventilator-free days among survivors, or both and assumes that any treatment that decreases the duration of ventilation among survivors also increases the number of survivors and that the decrease in duration of ventilation benefits the patient and care costs. Ventilator-free days permits smaller sample size if it is assumed that the treatment simultaneously reduces the duration of ventilation and improves mortality. It is unlikely that a treatment that led to higher mortality could lead to a statistically significant improvement in ventilator-free days, and this is especially true if the treatment were also required to produce a nominal improvement in mortality (29). Ventilator-free days are enumerated as the number of days from enrollment to day 28 during which a patient breathed without assistance, if this period of unassisted breathing lasted ≥48 consecutive hours. On day 28, survivors are assigned a score corresponding to the number of days they did not receive mechanical ventilation. Nonsurvivors are assigned zero ventilator-free days or a score corresponding to the number of days they were not supported with mechanical ventilation (as long as that was >48 hrs). This number is subtracted from the lesser of 28 or the number of days to death. Survivors supported with mechanical ventilation on day 28 are assigned a ventilator-free days score of 0. Ventilator days include bilevel positive airway pressure and continuous positive airway pressure of >5 cm H2O. For patients with tracheotomies, ventilator-free days are measured as the number of days after which a patient was returned to his or her pre-illness level of support. Ventilator days do not need to be contiguous—if the patient is transferred before day 28, it is assumed that on days for which no information is available, the subject was receiving mechanical ventilation. Again as ventilator-free days are not distributed normally, this variable is usually analyzed utilizing a Wilcoxon’s test, which weights mortality and duration of ventilation equally, but the Student’s t-test is valid for non-normal distributions when the sample size is large enough (29). Use of ventilator-free days in terms of trial design requires use of study protocols that decrease variation in ventilator management, extubation readiness testing, and ventilator weaning.
To control for patients who have not been discontinued from mechanical ventilation, but are probably ready to do so, an alternate physiologic end point, namely, time to recovery of acute lung injury, may be utilized. This variable reflects the number of days from randomization to achieving a pulmonary criterion, specifically, spontaneous breathing, weaning of mechanical ventilator support during the previous 24 hrs, and an oxygen index of ≤6 (27). Time to recovery of acute lung injury is calculated only for patients who recover, and if the criteria are not met by day 28, the case is censored. If transfer occurs before criteria are met, the patient is followed to compute the time to recovery of acute lung injury. It is assumed that once the patient is extubated, the oxygenation index is <6. Obviously, time to recovery of acute lung injury represents an example of a more generic physiologic end point, which is time to recovery of any event. A similar end point involves event-free survival, for example, defined as the time from enrollment into the study until the occurrence of an adverse event or until the last contact with the patient, whichever comes first. Adverse events might include disease progression, diagnosis of a second disease process, or death. Such end points are frequently utilized in hematology oncology clinical trials (30).
Evaluating Long-Term Outcomes.
Long-term outcome may be assessed as: functional health, an individuals ability to perform tasks of everyday life; quality of life, an individual’s subjective experience of the effect of health and treatment on one’s satisfaction with life; and health status, a combination of both functional health and quality of life. Various tools are available for assessing long-term outcome and require assessment of their validity, reliability, responsiveness, practicality, and whether they are available in the public domain. In terms of outcome assessment after critical illness in children, Pediatric Overall Performance Category and Pediatric Cerebral Performance Category scores have both been evaluated (31). Baseline, discharge, and delta Pediatric Overall Performance Category and Pediatric Cerebral Performance Category outcome scores have been associated with pediatric ICU length of stay and predicted risk of mortality (32). In addition, Pediatric Overall Performance Category is significantly related to the Stanford–Binet Intelligence Quotient, the Bayley Mental Developmental Index, and the Vineland Adaptive Behavior scores (33). For this outcome measurement, delta scores are calculated as the difference between baseline and discharge scores, which controls for the effect of existing conditions on final outcome. Typically, but not always, delta scores are positive, reflecting overall deterioration. This score has been validated in pediatric ICU populations and demonstrates face and content validity, criterion validity, and construct validity. These scores seem to be responsive to individual change over time. Interrater reliability is adequate as reflected by interclass coefficients that range from 0.88 to 0.96. This scale is easy to use, except for infants, for whom there may be significant overlap in the midrange scores. The Pediatric Overall Performance Category has been recommended by the American Academy of Pediatrics, American Heart Association, and the European Resuscitation Council for the reporting of outcomes associated with pediatric cardiopulmonary resuscitation (34, 35) and has been utilized in multiple studies (34–37).
Functional Health Assessment.
Again, multiple tools are available for this type of evaluation, but two are considered here, namely, WeeFIM® and Pediatric Evaluation of Disability Inventory. WeeFIM is the functional independence measure for children and is probably the most widely used scale for functional assessment. It measures severity of disability in terms of need for assistance in performing basic every-day life activities, for example, how much assistance is required for a child to complete each WeeFIM activity above and beyond that which is considered normal for age and what resources are consumed to maintain a particular quality of life for that child. Normative data are available, and using this assessment, most children achieve functional independence by 5–6 yrs of age. Validity, reliability, and responsiveness are well documented (38, 39). WeeFIM assesses six domains, namely, self-care, sphincter control, transfers, locomotion, communication, and social function, and is applicable for children from 6 months to 8 yrs of age and >8 yrs of age in children with developmental disabilities (40). Thirteen motor and five cognitive items are each rated on a seven-level ordinal scale that ranges from complete independence to complete dependence. The assessment is administered by parent-interviewer observation and requires <20 mins to complete. One limitation is that the WeeFIM is not sensitive for finer gradations of change in patients of <18 months of age. WeeFIM has been validated for use by direct observation or telephone interview (39, 41, 42) and has been accepted by the Joint Commission on Accreditation of Healthcare Organizations to provide performance measures for the Oryx initiative.
Measurement of long-term outcome can also be assessed using the Pediatric Evaluation of Disability Inventory, a discriminative measure of functional limitation (43, 44). It assesses three domains, namely, motor, self-care, and social function, and can be use to assess children between 6 months and 7.5 yrs of age. The Pediatric Evaluation of Disability Inventory is a 197-item test administered by parent interview or observation and requires approximately 45 mins to complete. Normative data are available, and validity and reliability are documented.
Challenges involved in quantifying quality of life include lack of tools that span the pediatric population, lack of premorbid quality-of-life baseline data, and lack of comparable populations (45). Recently, a benefit in terms of improved quality of life was demonstrated for antithrombin III in adult sepsis survivors in the KyberSept trial (46).
Clearly, long-term outcomes look beyond 28-day all-cause mortality. Obviously, the goal of treating a patient with severe sepsis is to return that individual to his or her premorbid health status. In this regard, there may be conflicts between treatments that are, on the one hand, life saving and, on the other, those that affect quality of life. For example, a long protective ventilation strategy for acute respiratory distress syndrome may reduce iatrogenic pulmonary injury and decrease mortality but might lead to a worse cognitive outcome. As nonsurvivors do not contribute data to studies of quality of life, treatments that increase mortality may preferentially lead to the death of debilitated patients who would have very poor quality of life if they had survived. Similarly, treatment that saves the lives of very debilitated patients may rescue these patients so that they are healthy enough to contribute very poor quality-of-life data to the study (47). Particular challenges to quality-of-life measurements in the pediatric population include the need for proxy responders, typically mothers or parents who live with the patient, leading to potential biases with parent-proxy measures. This may result in idealized views or expectations of a parent’s child, convergence of the parents’ subjective experiences with the perceived experience of their child, or transference of the parents’ own sense of debilitated function onto the child. An ideal quality-of-life instrument would include both child and parent-proxy report with concrete objective items (48). The age at which a child is able to give a valid and reliable response varies according to the type of information sought and the complexity of the questionnaire.
The Child Health Questionnaire measures overall health status of children 5–18 yrs of age (49). This tool utilizes 50 questions and generates two summary scores, namely, psychosocial and physical health. These summary scores are derived from 12 subscales. The psychometrics for the Child Health Questionnaire is suitable for use as a primary health-related quality-of-life outcome variable in clinical trials (50–53). This tool has been utilized after surgery for D-transposition of the great vessels (54).
Lastly, the Pediatric Quality of Life Inventory (PedsQL 4.0) is increasingly being utilized as a long-term outcome measure after pediatric critical illness. This is a modular measure of health-related quality of life, which is reliable in children aged 2–18 yrs and includes generic core scales (23 items) based on both child self-report and parent proxy. Four summary scores are generated, including physical, emotional, social, and school subscores. The Pediatric Quality of Life Inventory can be integrated with disease-specific measure, and psychometrics of the total score are suitable as a summary score for use as a primary health-related quality-of-life outcome variable in clinical trials (55–58).
Utilizing similar tools, long-term survival and state of health after pediatric intensive care have been examined in 254 children of >1 yr of age (59). In this cohort, ICU mortality was 7.5%, hospital mortality was 8.3%, and 1-yr mortality was 10.5%. In this investigation, the outcome assessment tool utilized was the Multiattribute Health Status Classification (60), which scores sensation, mobility, motion, cognition, self-care, and pain. The score was compared preadmission and 1 yr after discharge. Irrespective of the magnitude of change, overall health status improved in 25.7% and deteriorated in 27.4%. Most changes in overall health status were small, with only one domain more or less affected. Interestingly, no relation between acute severity of illness (Pediatric Risk of Mortality score) and health status before admission and after 1 yr were appreciated. Some degree of health impairment 1 yr after pediatric ICU discharge was present in 66.4% of patients—most had some health impairment before admission. Overall state of health 1 yr after discharge was unchanged or unimproved in 72.6% (59).
Intensive Care Unit Costs.
It would seem logical that intensity of illness would be related to intensity of therapeutic interventions and, accordingly, ICU costs. Utilizing care costs as an outcome measure in multiple-institution trials is complicated by regional variations in clinical practice, charges, and payor mix. However, these difficulties may be overcome by capitalizing on the relationship between the sum of Therapeutic Intervention Scoring System (TISS) (61, 62) scores and ICU costs. In fact, the correlation between total variable costs per patient admission episode and the total TISS per patient admission episode is high (63). The relationship between TISS and real pediatric ICU costs were examined in data collected from a ten-bed pediatric ICU over 17 months for children aged 1 month to 16 yrs. This cohort included 611 consecutive admissions and 3190 patient days. For this group of patients, it was feasible to calculate total direct medical costs based on a limited number of readily available clinical variables related to patient characteristics and treatment, of which TISS was the most important determinant (63). Physician time, nursing time, and pharmaceuticals contributed to 63% of the major direct medical cost components, and laboratory, material/disposables, and overhead/depreciation contributed 11%, 11%, and 10%, respectively. Although total patient costs per day were fairly normally distributed, total costs per admission were skewed to the left, with a long tail of higher costs associated with prolonged ICU admission for complex patients.
Resource utilization in the ICU has also been assessed with computerized TISS-based data (64). In this investigation, 1,229 adults were evaluated from >1,372 admissions in eight ICUs within the University of Pittsburgh system. TISS was collected manually, and a computerized TISS was obtained by developing a map between TISS items and corresponding charge items (transaction codes in the billing database). Mean and median manual and computerized TISS scores were nearly identical. Ninety-five percent of manually computed TISS scores were <10. Comparison between the computer-derived TISS score and the manually derived TISS score exactly matched on 55.5% of days and nearly matched on 83.3% of days, and correlation between the two was substantial (R2 = .85), with the highest correlation noted for scores of <10. Sensitivity, specificity, positive predictive value, and negative predictive value for mechanical ventilation and vasoactive drug infusion were all very high in terms of performance of the computerized TISS mapping algorithm.
It has been noted that the common pathologic event in sepsis is represented by the loss of the usual control mechanisms that regulate and compartmentalize inflammation (15). Accordingly, there has been wide interest in identifying reliable biomarkers, not only to detect sepsis but also to serve as a reliable surrogate in terms of sepsis resolution in response to a therapy. Interleukin 6 is known to be associated with sepsis mortality (65). In the trial of the assessment of safety and efficacy of monoclonal anti–tumor necrosis factor antibody fragment (MAK 195F) in patients with severe sepsis, patients with interleukin-6 levels of <1000 pg/mL showed similar response to varying doses of the anti–tumor necrosis factor antibody dosing. However, patients with initial interleukin-6 levels of >1000 pg/mL demonstrated a beneficial dose response effect in terms of mortality reduction with the anti–tumor necrosis factor intervention (66).
As a secondary outcome marker for goal-directed therapy in adults with severe sepsis, lactate clearance was utilized as a surrogate marker (67). Early sepsis resuscitation included maintaining central venous pressure at 10 ± 2, mean arterial pressure at >65, and urine output at >0.5 mL·kg−1·hr−1. For patients demonstrating ≥10% lactate clearance during their first few hours of resuscitation (as compared with <10%), a significant decrease in mortality was documented. Low lactate clearance is also a predictor of mortality in ICU patients with severe sepsis (68). These data confirm earlier studies indicating that time to clearance of lactate (lactime) was strongly associated with mortality (69). Pediatric sepsis mortality has also been examined as a function of blood lactate obtained 12 hrs after admission in 31 children with sepsis (70). At 24 hrs, a continuing lactate of >3 mM had a positive predictive value for death of 71%. Area under the receiver-operator curve plot for blood lactate assayed 12 hrs after admission was 0.81.
A biomarker of sepsis reflects a single biological aspect of a complex and heterogeneous process and not the entire spectrum of the disease. The systemic inflammatory response syndrome initiated by infection is complex, networked, redundant, and genetically determined and reflects microbial type and burden, concurrent illness, and acquired immunodeficiency (71). All of these contingencies need to be valued when considering a specific biomarker as a surrogate marker for sepsis clinical trials. For example, there seems to be an overwhelming production of nitric oxide in septic shock, which promotes peripheral vasodilation, induces catecholamine resistance, elicits cardiomyopathy, mediates cytopathic hypoxia, and inhibits vasopressin release (72). Multiple neonatal, pediatric, and adult investigations have demonstrated a clear relationship between the serum load of nitrate + nitrite (catabolic end products of nitric oxide) in relation to sepsis mortality and organ dysfunction (73–77). However, as noted above, an inhibitor substrate for nitric oxide synthase (546C88) reduced elevated plasma nitrate + nitrite concentrations observed in septic shock and improved vascular tone, easing successful maintenance of a target mean arterial blood pressure of >70 mm Hg with a reduction in vasoactive agents without adverse effects, but was associated with an increase in mortality (13, 14).
Sepsis continues to represent a major problem in both adult and pediatric populations. Probably related to various reasons for increasing acquired immunodeficiency, the prevalence of sepsis actually seems to be increasing. For a number of practical reasons, mortality as an end point for clinical trials of severe sepsis, particularly for pediatric studies, is not useful. Accordingly, there is an active search for meaningful, alternative surrogate markers. Currently, organ-failure–free days and long-term outcome are receiving the most scrutiny in pediatric clinical sepsis trials. An ideal biomarker that is sensitive and specific and reflects disease progression and resolution has not yet been validated.
1. Lefering R, Neugebauer EA: Steroid controversy in sepsis and septic shock: A meta-analysis. Crit Care Med
2. Cronin L, Cook DJ, Carlet J, et al: Corticosteroid treatment for sepsis: A critical appraisal and meta-analysis of the literature. Crit Care Med
3. Graf J, Doig GS, Cook DJ, et al: Randomized, controlled clinical trials in sepsis: Has methodological quality improved over time? Crit Care Med
4. de Keizer NF, Bonsel GJ, Al MJ, et al: The relation between TISS and real paediatric ICU costs: A case study with generalizable methodology. Intensive Care Med
5. Rubenfeld GD: Surrogate measures of patient-centered outcomes in critical care. In:
Surviving Intensive Care. Angus DC, Carlet J (Eds). Berlin, Springer-Verlag, 2002
6. Annane D, Sebille V, Charpentier C, et al: Effect of treatment with low doses of hydrocortisone and fludrocortisone on mortality in patients with septic shock. JAMA
7. Vincent JL: Endpoints in sepsis trials: More than just 28-day mortality? Crit Care Med
2004;32 (Suppl): S209–S213
8. Cohen JC, Guyatt G, Bernard GR, et al: New strategies for clinical trials in patients with sepsis and septic shock. Crit Care Med
9. Bernard GR: Sepsis trials: Intersection of investigation, regulation, funding and practice. Am J Respir Crit Care Med
10. Temple RJ: A regulatory authority’s opinion about surrogate endpoints. In:
Clinical Measurement in Drug Evaluation. Nimmo WS, Tucker GT (Eds). New York, J Wiley, 1995, p 329
11. Prentice RL: Surrogate endpoints in clinical trials: Definition and operational criteria. Stat Med
12. Bernard GR: Research in sepsis and acute respiratory distress syndrome: Are we changing course? Crit Care Med
13. Bakker J, Grover R, McLuckie A, et al: Administration of the nitric oxide synthase inhibitor NG
- methyl L-arginine hydrochloride (546C88) by intravenous infusion for up to 72 hours can promote resolution of shock in patients with severe sepsis: Results of a randomized, double-blind, placebo-controlled multicenter study (study no. 144-002). Crit Care Med
14. Watson D, Grover R, Anzueto A, et al: Cardiovascular effects of the nitric oxide synthase inhibitor NG
-methyl-L-arginine hydrochloride (546C88) in patients with septic shock: Results of a randomized, double blind, placebo-controlled multicenter study (study no. 144-002). Crit Care Med
15. Marshall JC, Vincent JL, Fink MP, et al: Measures, markers, and mediators: Toward a staging system for clinical sepsis. Crit Care Med
16. Giroir BP, Quint PA, Barton P, et al: Preliminary evaluation of recombinant amino-terminal fragment of human bactericidal/permeability-increasing protein in children with severe meningococcal sepsis. Lancet
17. Levin M, Quint PA, Goldstein B, et al: Recombinant bactericidal/permeability increasing protein (rBPI21
) as adjunctive treatment for children with severe meningococcal sepsis: A randomized trial. Lancet
18. Giroir BP, Scannon PJ, Levin M: Bactericidal permeability-increasing protein: Lessons learned from the phase III, randomized, clinical trial of rBPI21
for adjunctive treatment of children with severe meningococcemia. Crit Care Med
2001;29 (Suppl): S130–S135
19. Bota DP, Melot C, Lopes Ferreira F, et al: The Multiple Organ Dysfunction Score (MODS) versus the Sequential Organ Failure Assessment (SOFA) score in outcome prediction. Intensive Care Med
20. Wilkinson JD, Pollack MM, Ruttiman UE, et al: Outcome of pediatric patients with multiple organ system failure. Crit Care Med
21. Wilkinson JD, Pollack MM, Glass NL, et al: Mortality associated with multiple organ system failure and sepsis in pediatric intensive care unit. J Pediatr
22. Proulx F, Gauthier M, Nadeau D, et al: Timing and predictors of death in pediatric patients with multiple organ system failure. Crit Care Med
23. Tantalean JA, Leon RJ, Santos AA, et al: Multiple organ dysfunction in children. Pediatr Crit Care Med
24. Leteurtre S, Martinot A, Duhamel A, et al: Validation of the paediatric logistic organ dysfunction (PELOD) score: Prospective, observational, multicenter study. Lancet
25. Vincent JL, Angus DC, Artigas A, et al: Effects of drotrecogin alfa (activated) on organ dysfunction in the PROWESS trial. Crit Care Med
26. Goldstein B, Giroir B, Randolph A, et al: International Pediatric Severe Sepsis Consensus Conference: Definitions for sepsis and organ dysfunction in pediatrics. Pediatr Crit Care Med
27. Curley MAQ, Fineman LD, Cvijanovich N, et al: Prone positioning in pediatric acute lung injury: Results of a randomized controlled clinical trial. Presented at the 15th ESPNIC Medical and Nursing Annual Congress, London, September, 2004
28. Wilson DF, Zaritsky A, Bauman LA, et al: Instillation of calf lung surfactant extract (calfactant) is beneficial in pediatric acute hypoxemic respiratory failure. Crit Care Med
29. Schoenfeld DA, Bernard GR: Statistical evaluation of ventilator-free days as an efficacy measure in clinical trials of treatments for acute respiratory distress syndrome. Crit Care Med
30. Grier HE, Krailo MD, Tarbell NJ, et al: Addition of ifosfamide and etoposide to standard chemotherapy for Ewing’s sarcoma and primitive neuroectodermal tumor of bone. N Engl J Med
31. Fiser DH: Assessing outcome of pediatric intensive care. J Pediatr
32. Fiser DH, Tilford JM, Robertson PK: Relationship of illness severity and length of stay to functional outcomes in the pediatric intensive care unit: A multi-institutional study. Crit Care Med
33. Fiser DH, Long N, Robertson PK, et al: Relationship of pediatric overall performance category and pediatric cerebral performance category scores at pediatric intensive care unit discharge with outcome measures collected at hospital discharge and 1- and 6-month follow-up assessments. Crit Care Med
34. Schindler MB, Bohn D, Cox PN, et al: Outcome of out-of-hospital cardiac or respiratory arrest in children. N Engl J Med
35. Torres A Jr, Pickert CB, Firestone J, et al: Long-term functional outcome of inpatient pediatric cardiopulmonary resuscitation. Pediatr Emerg Care
36. Hackbaarth RM, Rzeszutko KM, Sturm G, et al: Survival and functional outcome in pediatric traumatic brain injury: A retrospective review and analysis of predictive factors. Crit Care Med
37. Goldstein B, Fiser DH, Kelly MM, et al: Decomplexification in critical illness and injury: Relationship between heart rate variability, severity of illness, and outcome. Crit Care Med
38. Ziviani J, Ottenbacher KJ, Shephard K, et al: Concurrent validity of the Functional Independence Measure for Children (WeeFIM) and the Pediatric Evaluation of Disabilities Inventory in children with developmental disabilities and acquired brain injury. Phys Occup Ther Pediatr
39. Sperle PA, Ottenbacher KJ, Braun SL, et al: Equivalence reliability of the Functional Independence Measure for Children (WeeFIM): Administration methods. Am J Occup Ther
40. WeeFIM System Clinical G uide: Version 5. 01. Buffalo, NY, State University of New York at Buffalo, 1998
41. Msall ME, DiGaudio K, Duffy LC: Use of functional assessment in children with developmental disabilities. Phys Med Rehabil Clin North Am
42. Ottenbacher KJ, Taylor ET, Msall ME, et al: The stability and equivalence reliability of the Functional Independence Measure for Children (WeeFIM). Dev Med Child Neurol
43. Berg M, Jahnsen R, Froslie KF, et al: Reliability of the Pediatric Evaluation of Disability Inventory (PEDI). Phys Occup Ther Pediatr
44. Kothari DH, Haley SM, Gill-Body KM, et al: Measuring functional change in children with acquired brain injury (ABI): Comparison of generic and ABI-specific scales using the Pediatric Evaluation of Disability Inventory (PEDI). Phys Ther
45. Herridge MS, Cheung AM, Tansey CM, et al: One-year outcomes in survivors of the acute respiratory distress syndrome. N Engl J Med
46. Rublee D, Opal SM, Schramm W, et al: Quality of life effects of antithrombin III in sepsis survivors: Results from the KyberSept trial [ISRCTN22931023]. Crit Care
47. Rubenfeld G: Looking beyond 28-day all-cause mortality. Crit Care
48. Parsons SK, Brown AP: Evaluation of quality of life of childhood cancer survivors: A methodological conundrum. Med Pediatr Oncol
49. Landgraf JM, Abetz L, Ware JE: Childhealth Questionnaire (CHQ) Users Manual. First Edition. Boston, Health Institute, New England Medical Center, 1996
50. Georgalas C, Tolley N, Kanagalingam J: Measuring quality of life in children with adenotonsillar disease with the Child Health Questionnaire: A first UK study. Laryngoscope
51. Panepinto JA, O’Mahar KM, DeBaun MR, et al: Validity of the child health questionnaire for use in children with sickle cell disease. J Pediatr Hematol Oncol
52. Warschburger P, Landgraf JM, Petermann F, et al: Health-related quality of life in children assessed by their parents: Evaluation of the psychometric properties of the CHQ-PF50 in two German clinical samples. Qual Life Res
53. Klassen AF, Landgraf JM, Lee SK, et al: Health related quality of life in 3 and 4 year old children and their parents: Preliminary findings about a new questionnaire. Health Qual Life Outcomes
54. Dunbar-Masterson C, Wypij D, Bellinger DC, et al: General health status of children with D-transposition of the great arteries after the arterial switch operation. Circulation
2001;104(12 Suppl 1):I138–I142
55. Varni JW, Seid M, Kurtin PS: PedsQL 4.0: Reliability and validity of the Pediatric Quality of Life Inventory version 4.0 generic core scales in healthy and patient populations. Med Care
56. Varni JW, Seid M, Rode CA: The PedsQL in pediatric rheumatology: Reliability, validity, and responsiveness of the Pediatric Quality of Life Inventory Generic Core Scales and Rheumatology Module. Arthritis Rheum
57. Varni JW, Burwinkle TM, Katz ER, et al: The PedsQL in pediatric cancer: Reliability and validity of the Pediatric Quality of Life Inventory Generic Core Scales, Multidimensional Fatigue Scale, and Cancer Module. Cancer
58. Varni JW, Seid M, Knight TS, et al: The PedsQL 4.0 Generic Core Scales: Sensitivity, responsiveness, and impact on clinical decision-making. J Behav Med
59. Gemke RJBJ, Bonsel GJ, van Vught AJ: Long term survival and state of health after paediatric intensive care. Arch Dis Child
60. Feeny D, Furlong W, Barr RD, et al: A comprehensive multiattribute system for classifying the health status of survivors of childhood cancer. J Clin Oncol
61. Cullen DJ, Civetta JM, Briggs BA, et al: Therapeutic intervention scoring system: A method for quantitative comparison of patient care. Crit Care Med
62. Pollack MM, Yeh TS, Ruttiman UE, et al: Evaluation of pediatric intensive care. Crit Care Med
63. Dickie H, Vedio A, Dundas R, et al: Relationship between TISS and ICU cost. Intensive Care Med
64. Clermont G, Angus DC, Linde-Zwirble WT, et al: Measuring resource use in the ICU with computerized Therapeutic Intervention Scoring System–based data. Chest
65. Damas P, Ledoux D, Nys M, et al: Cytokine serum levels during severe sepsis in humans: IL-6 as a marker of severity. Ann Surg
66. Reinhart K, Weigand-Löhnert C, Grimminger F, et al: Assessment of the safety and efficacy of the monoclonal anti-tumor necrosis factor antibody-fragment, MAK 195F, in patients with sepsis and septic shock: A multicenter, randomized, placebo-controlled, dose-ranging study. Crit Care Med
67. Nguyen HB, Rivers EP, Knoblich BP, et al: Early lactate clearance is associated with improved outcome in severe sepsis and septic shock. Crit Care Med
68. Levraut J, Ichai C, Petit I, et al: Low exogenous lactate clearance as an early predictor of mortality in normolactatemic critically ill septic patients. Crit Care Med
69. Bakker J, Gris P, Coffernils M, et al: Serial blood lactate levels can predict the development of multiple organ failure following septic shock. Am J Surg
70. Duke TD, Butt W, South M: Predictors of mortality and multiple organ failure in children with sepsis. Intensive Care Med
71. Zimmerman JJ: A question of balance. Crit Care Med
72. Vincent JL, Zhang H, Szabo C, et al: Effects of nitric oxide in septic shock. Am J Respir Crit Care Med
73. Yuan S, Li HQ, Shen CK, et al: Plasma nitric oxide levels in newborn infants with sepsis. J Pediatr
74. Doughty LA, Kaplan SS, Carcillo JA: Inflammatory cytokine and nitric oxide responses in pediatric sepsis and organ failure. Crit Care Med
75. Wong HR, Carcillo JA, Burckart G, et al: Increased serum nitrite and nitrate concentrations in children with the sepsis syndrome. Crit Care Med
76. Kraft-Jacobs B, Brilli R, Szabo C, et al: Circulating methemoglobin and nitrite/nitrate concentrations as indicators of nitric oxide overproduction in critically ill children with septic shock. Crit Care Med
77. Spack L, Havens PL, Griffith OW: Measurements of total plasma nitrite and nitrate in pediatric patients with the systemic inflammatory response syndrome. Crit Care Med