KEY WORDS: severity of illness index; mortality prediction; pediatrics; critical illness; patient outcome assessment; intensive care unit, pediatric
Severity of illness assessment has been crucial for a wide range of pediatric, neonatal, and adult intensive care unit (ICU) uses, including quality assessments, controlling for severity of illness in clinical studies, and studies of ICU resource utilization and management [1-6]. Although severity of illness is a familiar medical concept, it is sometimes difficult to define. In the context of intensive care, a rational and objective way to define and quantify severity of illness is through the development of probability models predicting mortality risk . Such predictive models have been developed for all age groups [8-13]. Future uses of outcome probabilities may even include decision-making for individual patients, if predictors achieve a sufficient level of accuracy and validity .
The relationship between physiologic status and mortality risk may change as new treatment protocols, therapeutic interventions, and monitoring strategies are introduced. Patient populations may also change as new therapies ameliorate the requirement for ICU care, and new patient groups may emerge, often as a result of other medical advances. Predictive models evolve as databases become larger and additional patient characteristics can be integrated into the predictive algorithms.
The Pediatric Risk of Mortality (PRISM) is a second-generation, physiology-based predictor for pediatric ICU patients. PRISM was initially derived from the Physiologic Stability Index [8,15]. The goal of the present study was the development and validation of PRISM III, a third-generation score, based on a sample of 11,165 admissions to 32 pediatric ICUs, representing a wide diversity of organizational and structural characteristics. Specifically, the physiologic variables and their ranges, as well as diagnostic and other risk variables reflective of mortality risk, were reevaluated to up-data and improve the performance of the score. In addition, since minimizing the time period for assessing mortality risk is advantageous for evaluating pediatric ICU quality, we developed a 12-hr prediction model as well as a 24-hr prediction model. Concepts that guided this effort included the following: a) maximizing the predictive performance while keeping the number of variables and their ranges to a minimum, using variables that are readily available and clearly definable while maintaining the assumptions inherent in the Physiologic Stability Index and PRISM that unmeasured variables are assumed to be normal; and b) avoidance of therapeutic variables that may be unduly influenced by practice patterns.
MATERIALS AND METHODS
There were 32 study sites. The selection process for the first 16 units has been previously reported [1,16]. A stratified sample of pediatric ICUs representing a broad range of organizations and structures was randomly selected based on size, unit coordination, presence or absence of a pediatric intensivist, and teaching status of the hospital. In addition, a data set from 18 volunteer units were collected in 1993 and 1994, although two units were excluded because they did not meet criteria for data reliability. The characteristics of these units are shown in Table 1.
Consecutive admissions to each pediatric ICU were included, unless they met the criteria for exclusion specified below. Readmissions to the pediatric ICU during the same hospitalization were analyzed as separate patients because each admission presented a separate opportunity for a pediatric ICU outcome. Excluded from the study were: a) admissions for recovery from procedures normally cared for in other hospital locations; b) patients staying in the ICU less than 2 hrs; c) patients transferred from the study pediatric ICU to another ICU because their outcome could not be clearly credited to either ICU; and d) patients admitted in a state of continuous cardiopulmonary resuscitation who never achieved stable vital signs for at least 2 hrs. If deaths occurred in the operating room, the patients were included if the operation occurred during the pediatric ICU stay and was a therapy for the illness requiring pediatric ICU care. Terminally ill patients who were transferred from the pediatric ICU for ``comfort care'' after discontinuation of a pediatric ICU technology (e.g., mechanical ventilation) were included as pediatric ICU patients for the 24 hrs after pediatric ICU discharge because 24 hrs is a routine observational time after technology is discontinued. Terminally ill patients transferred from the pediatric ICU for comfort care while technological support was maintained were included as pediatric ICU patients until 24 hrs after the technological support was discontinued. Patients transferred out of the pediatric ICU with technological support who were not considered terminal (e.g., chronic mechanical ventilation) were classified as survivors.
All institutions collected information on all admissions. When the last death in each pediatric ICU's sample occurred, all patients admitted before that death remained in the study. All pediatric ICUs submitted patient logs. These logs were assessed to ensure that no deaths were left out, data on at least 97% of patients were included, and none of the patients who lacked data died.
In the first 16 pediatric ICUs, data collection methods were taught at site visits. In the volunteer pediatric ICUs, a video tape teaching program was used. For both groups, a detailed protocol manual was supplied. Patient data included the following information: age; gender; pediatric ICU and hospital outcomes (survival, death); admission and discharge diagnoses classified by system and etiology of disease; elective/emergency status; operative status; clinical service of primary responsibility; admission source (same hospital nursing unit, referral hospital nursing unit, home, physician office/clinic); transportation to hospital by an organized transport system (helicopter, fixed wing, ambulance, none); previous pediatric ICU admission during the current hospitalization; cardiac massage before the pediatric ICU or hospital admission; and selected critical care modalities used in the first 24 hrs of the pediatric ICU stay. In addition to the diagnostic classification using system and etiology of disease, we also investigated a more traditional diagnostic system, using the common diagnoses (asthma, pneumonia, meningitis, seizures, head trauma, other trauma, human immunodeficiency virus status, congenital heart disease, diabetes, sepsis, and bronchopulmonary dysplasia). Diagnoses were determined from admission-day information.
Physiologic data included the most abnormal values from the first 12 hrs and the second 12 hrs of pediatric ICU stay. In the first 16 units, data collection involved obtaining photocopies of the vital sign and laboratory records, and the appropriate items were extracted at the data center. In the second group of units, the data were collected at the sites. The data consisted of the following: systolic and diastolic blood pressures; heart rate; respiratory rate; temperature (oral, axillary, or core); coma status; pupillary reactions; pupillary size and equality; concentrations of sodium, potassium, total CO2, bicarbonate, total and direct bilirubin, total and ionized calcium, glucose, blood urea nitrogen, creatinine, and albumin; hemoglobin; white blood cell count; platelet count; prothrombin and partial thromboplastin times; pH and PCO2 (arterial, venous, or capillary); and PaO2 with a simultaneous FIO2. Whole blood as well as serum/plasma measurements of sodium, potassium, and glucose were also collected. For variables where both high and low abnormalities may reflect increased mortality risk, we collected both the high and the low values. Thus, both high and low values of the same physiologic variable could contribute to severity of illness. Heart rate, respiratory rate, and blood pressure were not included at times when crying or iatrogenic agitation was noted. Physiologic data accumulated during the preterminal period in patients dying within the first 24 hrs of pediatric ICU care were not included in the study when death was obvious (usually, the last 2 to 4 hrs of life).
Since altered mental status can be influenced by a variety of iatrogenic interventions, we only considered mental status for children with known acute central nervous system disease, or where acute central nervous system disease secondary to an acute, systemic event (e.g., hypoxia, hypotension) was a possibility. In addition, we did not include mental status assessments for the 2 hrs after sedatives, paralyzing drugs, or anesthetic agents. If patients were sedated or paralyzed during the entire assessment period, the mental status assessment most proximate to pediatric ICU admission without sedation, paralysis, or anesthesia was used (usually in the emergency department). Altered mental status was defined as a Glasgow Coma Scale score of less than 8, or stupor or coma.
Physiologic variables, where normal physiologic values are age dependent, were stratified into the following age groups: neonates (less than 1 month); infants (more than equals 1 to 12 months); children (more than equals 12 to 144 months); and adolescents (more than 144 months). Age-adjusted variables included the following: systolic blood pressure; diastolic blood pressure; heart rate; respiratory rate; concentrations of blood urea nitrogen, creatinine, albumin, and bilirubin; hemoglobin, prothrombin time, partial thromboplastin time, and PaO2.
When several variables overlapped significantly in the assessment of physiologic dysfunction, we attempted to combine them into a composite variable. This approach was most pertinent for acidosis variables and clotting variables. For example, we combined pH and total CO2 into a variable representing acidosis.
The reliability of the data collection, entry, and verification processes were formally checked by reabstracting a random selection of at least 23 cases from each institution after completion of the initial data collection. The reabstractions were subjected to the identical processes of data entry and verification, and PRISM scores were recalculated. Institutions were included if the intraclass correlation coefficient of reliability  for their abstraction/reabstraction of PRISM scores was more than 0.80, resulting in the exclusion of two of the volunteer pediatric ICUs.
Variable and Range Selection.
Initially, we developed separate prediction models for two time periods, one for the first 12 hrs and one for the first 24 hrs of pediatric ICU stay. Our approach to this portion of PRISM III development assumed that deviations of physiologic variables from the midrange (40th to 60th percentiles) of survivors positively contributed to mortality risk, with larger deviations reflecting higher mortality risks. Appropriate variable ranges that significantly contributed to mortality prediction were investigated initially using univariate logistic regression analysis. The risk of death (odds ratios) relative to the midrange of survivors was computed for each physiologic variable. Continuous physiologic variables were initially subdivided into ranges based on percentiles of survivors (5%, 10%, 20%, 30%, 40% to 60%, 70%, 80%, 90%, 95%). In some instances, the resulting cutoff points were modified based on clinical judgment. The variable ranges were absorbed into the midrange under the following conditions: a) the logistic regression coefficients of the variable ranges were not significant (p more than equals .25) and they bordered the midrange or a range that had been combined with the midrange; or b) none of the deaths had variable values in midrange. Such a variable range was then combined with the range displaying the most similar regression coefficient. When the regression coefficients of two or more adjacent ranges were within the standard errors, the ranges were also combined.
These univariate procedures yielded 21 physiologic variables with 78 ranges for inclusion into the multivariate logistic model. Table 2 illustrates the ranges for one of the variables, systolic blood pressure. The logistic model utilized a stepwise variable inclusion procedure . The ranges of the predictor variables were included in the logistic regression model, one at a time, as long as the Akaike Information Criterion decreased. Subsequently, to obtain the best subset of ranges for each variable and, at the same time guard against overfitting, we employed a cross-validation by a repeated training-testing method . For that purpose, ten random validation samples, each consisting of 10% of the total sample, were generated without replacement. The ten complementary 90% samples served as training samples to develop the ``best'' model for that sample, based on the minimum Akaike Information Criterion. The associated 10% sample was used as the test sample for validating the corresponding model. For each of the ten resulting ``best'' models, model fit was assessed by computing the mean square deviance in both the training and the validation samples . The final model for the physiologic variables was selected from the ten ``best'' candidates as the one that displayed the maximum difference between the mean square deviance values in the associated training and validation samples, while the Hosmer-Lemeshow goodness-of-fit test was not significant at a level of p more than .10 . This process maximally separated the training and validation samples with respect to the prediction performance of the model, enabling the testing of the most deviant validation sample for goodness-of-fit. After the selection of the final physiologic model, the logistic regression coefficients were scaled to yield integer scores for the individual variable ranges. The sum of these scores constitutes PRISM III.
After the development of the physiologic portion of the PRISM score, diagnostic and other risk variables were tested for effect on mortality prediction. The association of these risk variables with outcome was assessed by multivariate logistic regression analyses in the previously selected ten training samples, with PRISM III as a covariate in the model. Variable inclusion was based on minimizing the Akaike Information Criterion value in each sample. The final model was chosen from the best training sample models that included only variables selected in the majority of the training samples and yielded the highest prediction accuracy, while maintaining the Hosmer-Lemeshow goodness-of-fit test with p more than .10 in both the training and validation samples. The goodness-of-fit test assessed model calibration, while prediction accuracy was measured by the area under the receiver operating characteristic curve . Model fit in the validation sample was also assessed by Flora's method .
Finally, the performance of the previous version of the PRISM physiology score was compared directly with PRISM III by using the variables (the physiologic score, age, and operative status) and observation period (24 hrs) as specified by the previous version of PRISM. Improvements in the Akaike Information Criterion and the loglikelihood ratio were compared using the percentage improvement in the training set. The area under the receiver operating curve was compared in both the training and validation sets using Hanley's method .
Data were collected on 11,165 admissions (543 deaths). In the first data set of 16 pediatric ICUs, there were 5,415 admissions; in the second data set of 16 volunteer pediatric ICUs, there were 5,750 admissions. Population characteristics are summarized in Table 1.
Multivariate logistic regression modeling resulted in a PRISM III score based on the first 12 hrs of care, consisting of 17 physiologic variables subdivided into 26 ranges, and a PRISM III score based on the first 24 hrs of care, consisting of 17 physiologic variables subdivided into 26 ranges. The variables selected for the first 12 hrs and first 24 hrs were identical, with the exception of potassium concentration, which was included in the first 12-hr score but not in the first 24-hr score, and respiratory rate, which was included in the first 24-hr score but not in the first 12-hr score. The ranges and their relative contributions to risk prediction scores were almost identical. Subsequent analysis demonstrated similar performance between the PRISM III score specifically determined from the first 24 hrs applied to the first 12 hrs, and the PRISM III score determined from the first 12 hrs applied to the first 24 hrs. Therefore, only a single set of physiologic variables and ranges derived from the first 12 hrs was used for determining physiologic status in both the first 12 hrs and first 24 hrs of care (PRISM III, Figure 1). The PRISM III score, when obtained from the first 12 hrs, is denoted as PRISM III-12. The PRISM III score, when obtained from the first 24 hrs, is denoted as PRISM III-24. Five physiologic variables are age adjusted. For some variables (e.g., systolic blood pressure), different physiologic ranges are used for each age group, while for other variables (e.g., partial thromboplastin time), several of the age groups share the same physiologic ranges. When both high and low ranges are included for a physiologic variable (e.g., pH), PRISM points may be assigned for both the high and the low range if abnormalities in both ranges occur. The variables representing acidosis and coagulation are composite variables, combining in an ``either/or'' format the most extreme deviation of either variable. This structure worked as well as more complicated variable combination schemes and was simpler to use. Data collection rules have been provided in the Notes at the end of Figure 1.
The variables that were most predictive of mortality, as indicated by the highest PRISM III scores, were minimum systolic blood pressure, abnormal pupillary reflexes, and stupor/coma. Variables in the original PRISM that are not included in PRISM III are diastolic blood pressure, respiratory rate, PaO2 /FIO2, and bilirubin and calcium concentrations. Variables that are included in PRISM III but not in PRISM are temperature, pH, PaO2, creatinine concentration, blood urea nitrogen concentration, white blood cell count, and platelet count.
After selection of the PRISM III physiologic variables and their ranges, additional predictive factors were tested for their effects on mortality prediction by building logistic regression models with either PRISM III-12 or PRISM III-24 as a covariate. This approach resulted in the inclusion of a PRISM III squared term , two acute diagnoses (diabetes and nonoperative cardiovascular disease), two diagnoses reflecting acute and chronic health status (chromosomal anomalies, oncologic disease), and four additional risk variables reflecting pre-ICU risk factors (operative status, pre-ICU care area, pre-ICU cardiac massage, and previous ICU admissions) Table 3. Table 4 illustrates the goodness-of-fit data for the PRISM III-12 model, with all significant risk variables. Figure 2 illustrates the observed and expected mortality rates for all training models. Overall, the additional risk variables contributed 5% to the variance explained by the models, while PRISM III contributed 95%. Figure 3 shows the receiver operating characteristic curves.
The performance of the predictors in the validation sample is shown in Table 5. For all models, the Hosmer-Lemeshow chi-square and Flora's z-statistic indicated excellent fit in this independent sample, although generally, the PRISM III-24 model performed better than the PRISM III-12 model. Table 6 shows the goodness-of-fit to the validation data for the best performing model: PRISM III-24 with the additional diagnostic and risk factors. Figure 4 illustrates the observed and expected mortality rates for all of the validation models.
Two additional goodness-of-fit evaluations were done, using the total sample to assess model calibration for different patient groups. First, patients were stratified by the major diagnostic categories causing death, and the full PRISM III-12 and PRISM III-24 models were tested. In both cases, the fit was excellent (PRISM III-12: chi-square, 6 degrees of freedom equals 4.576, p equals .5992; PRISM III-24: chi-square, 6 degrees of freedom equals 3.118, p equals .7939). Table 7 shows the data for the PRISM III-12 model. The performance in the age groups was similarly tested and both full models performed well (PRISM III-12: chi-square, 4 degrees of freedom equals 6.541, p equals .1622; PRISM III-24: chi-square, 4 degrees of freedom equals 3.944, p equals .4137). The performance of the PRISM III-24 model for the different age groups is shown in Table 8.
Finally, PRISM III-24 was compared with the original PRISM, as described in the Materials and Methods section. In the training set, the Akaike Information Criterion improved by 18.4% (PRISM 2214.23; PRISM III-24 1807.749), the -2ln(likelihood ratio) improved by 24.4% (PRISM 1663.051; PRISM III 2069.533), and area under the receiver operating curve improved by 3.9% (PRISM 0.914; PRISM III 0.950, p less than .0001). In the validation set, the area under the receiver operating curve was also significantly improved by 9.0% (PRISM 0.831; PRISM III 0.906, p less than .0005).
The development of PRISM III resulted in several improvements over the original PRISM. First, the physiologic variables and their ranges were reevaluated. The variables and the ranges in PRISM had been originally selected based on the subjective opinions of physicians who developed the Physiologic Stability Index. When the PRISM score was developed from these variables, objectivity was added, but a reevaluation of the original ranges was not undertaken. In this study, we objectively reassessed the predictive power of the physiologic variables and their ranges, eliminating some ranges that did not contribute significantly to mortality risk (e.g., high systolic blood pressure), and revising the ranges of the retained physiologic variables. Some physiologic variables have been eliminated and others--including temperature, pH, PaO2, creatinine concentration, blood urea nitrogen concentration, white blood cell count, and platelet count-have been added. Although these are important changes, the variables with the greatest importance in outcome prediction are the same in both scores: low systolic blood pressure, altered mental status, and abnormal pupillary reflexes.
Second, age issues, clear data collection instructions, precise variable definitions, and strict rules for patient inclusions and exclusions were addressed at the outset of this study. While age was included as an explicit variable in the original PRISM score, it is included in the PRISM III score in a logically and clinically more convincing form by using appropriate age-adjusted physiologic variable ranges. Subsequent model fit evaluations demonstrated the success of these adjustments. A formal operational method for assessing mental status also was established to account for the frequent use of sedation and paralysis. Other variables included in the prediction model are better defined, making the score less vulnerable to ``gaming.'' Two diagnostic entities, chromosomal abnormalities and oncologic disease, reflect underlying health status as well as acute disease status. Two acute diagnoses include nonoperative cardiovascular disease and acute diabetes (primarily diabetic ketoacidosis). Other risk factors include operative status, pre-ICU care area, pre-ICU cardiac massage, and previous ICU admission.
Third, the relationship between physiologic status, as measured by PRISM III, and outcomes has been calibrated to a contemporary, well-defined, large reference sample. The set of 32 pediatric ICUs represents about 10% of all pediatric ICUs in the United States. These units encompass a wide diversity of organizational structure and patient mixes. This diversity makes the sample sufficiently representative for most units, enabling PRISM III to be used in the comparative assessment of pediatric ICU outcomes in essentially all pediatric ICUs.
Our method of developing the PRISM III models continued the evolution toward a parsimonious predictor. The Physiologic Stability Index incorporated 102 discrete physiologic ranges of 34 physiologic variables selected by physicians for their clinical importance. PRISM reduced the number of physiologic variables to 14 and their ranges to 34. While PRISM III added several new variables, the total number of ranges was reduced. Differences in the frequency of measuring variables associated with individual pediatric ICUs are unlikely to influence the reliability or accuracy of PRISM III . An alternative approach of including more physiologic ranges could have been accomplished by applying less strict statistical criteria for variable and range inclusion. However, this approach may have increased the variability of the predictor, decreasing the power of detecting truly existing differences from the expected mortality rates. More importantly, it could produce a biased (``overfitted'') model that might perform very well in the training sample but poorly in an independent sample by incorporating idiosyncrasies of the training sample, and thus, may be biased. The excellent performance in the training sample may generate an unjustified confidence in the predictor's prediction accuracy.
Overall, all PRISM III prediction models were accurately calibrated and achieved good discrimination. The PRISM III-24 model with the diagnostic and other risk variables performed best. This result was expected, since PRISM III-24 incorporates the most information over the longest time period. However, the other models also performed very well and are suitable for quality assessment. We recommend using the PRISM III models with the additional variables since these models may increase the applicability to a wider variety of case-mix samples. The use of the PRISM III-12 model is appealing for quality assessments since, by shortening data acquisition time, it better separates the observation from the treatment period, while the PRISM III-24 model is more accurate for individual patient mortality risk assessments.
As expected, PRISM III performed better than PRISM, even when limited to the variables originally included in PRISM. The improvement in the area under the receiver operating curve was similar to the improvement seen with more recent versions of adult severity scores compared with their previous versions . Newer versions of severity of illness scores, such as PRISM III, will need revisions and recalibrations to maintain their relevance to contemporary patient populations.
This study was, in part, an independent effort of members of the National Association of Children's Hospitals and Related Institutions. The following study sites and institutional coordinators participated: Mark E. Swanson, MD, Arnold Palmer Hospital for Children and Women, Orlando, FL; Jacob Hen, Jr, MD, Bridgeport Hospital, Bridgeport, CT; Bob Lynch, MD, Cardinal Glennon Children's Hospital, Saint Louis, MO; James Fackler, MD, Children's Hospital, Boston, MA; Barbara A. Jackson, MD, and Stephen Levine, MD, Children's Hospital, New Orleans, LA; Mary W. Lieh-Lai, MD, Children's Hospital of Michigan, Detroit, MI; Frank Allman, MD, Children's Hospital Medical Center of Akron, Akron, OH; William A. Spohn, MD, The Children's Medical Center, Dayton, OH; James D. Wilkinson, MD, Children's National Medical Center, Washington, DC; Tara Snellgrove, MD, and Linda Lai, RN, Cook Children's Medical Center, Fort Worth, TX; Peter Quint, MD, Emanuel Children's Hospital and Health Center, Portland, OR; Suresh Havalad, MD, Lutheran General Hospital, Park Ridge, IL; David M. Habib, MD, Medical University of South Carolina, Charleston, SC; Kathleen Winder, RN, Memorial Hospital, Colorado Springs, CO; Linda Marzano, MD, Miami Children's Hospital, Miami, FL; Suzanne Sander, RN, and Stephen C. Kurachek, MD, Minneapolis Children's Medical Center, Minneapolis, MN; Richard M. Ruddy, MD, New York Medical Center, Valhalla, NY; Rex Northup, MD, Sacred Heart Hospital, Pensacola, FL; Maggie Halley, MD, Saint Joseph's Hospital and Medical Center, Phoenix, AZ; David L. Peterman, MD, Saint Luke's Regional Medical Center, Boise, ID; James H. Jose, MD, Scottish Rite Children's Medical Center, Atlanta, GA; Gary A. Neidich, MD, Sioux Valley Hospital, University of South Dakota School of Medicine, Sioux Falls, SD; Marjory K. Waterman, RN, MN, Southwest Texas Methodist Hospital, San Antonio, TX; Maria Cox, RN, Tampa Children's Hospital, Tampa, FL; Wallace W. Marsh, MD, and John Cochran, MD, Texas Tech University Health Science Center and University Medical Center Hospital, Lubbock, TX; Arno Zaritsky, MD, University of North Carolina, Chapel Hill, NC.
The authors also wish to acknowledge the individual efforts of Timothy Cuerdon, PhD, Marcia Levetown, MD, Walter Clark, BA, Kia Banks, BA, and Joyce Sheppard, RN.
1. Pollack MM, Cuerdon TC, Patel KM, et al: Impact of quality-of-care factors on pediatric intensive care unit mortality. JAMA 1994; 272:941-946
2. Pollack MM, Getson PR, Ruttimann UE, et al: Efficiency intensive care. A comparative analysis of eight pediatric intensive care units. JAMA 1987; 258:1481-1486
3. Knaus WA, Wagner DP, Zimmerman JE, et al: Variations in mortality and length of stay in intensive care units. Ann Intern Med 1993; 118:753-761
4. Shortell SM, Zimmerman JE, Rousseau D, et al: The performance of intensive care units: Does management make a difference? Med Care 1994; 32:508-525
5. Rapoport J, Teres D, Lemeshow S, et al: A method for assessing the clinical performance and cost-effectiveness of intensive care units: A multicenter inception cohort study. Crit Care Med 1994; 22:1385-1391
6. Pollack MM, Alexander SR, Clarke N, et al: Improved outcomes from tertiary center, pediatric intensive care: A statewide comparison of tertiary and nontertiary care facilities. Crit Care Med 1991; 19:150-159
7. Lemeshow S, Le Gall JR: Modelling the severity of illness of ICU patients. JAMA 1994; 272:1049-1055
8. Pollack MM, Ruttimann UE, Getson PR: The Pediatric Risk of Mortality (PRISM) Score. Crit Care Med 1988; 16:1110-1116
9. Le Gall JR, Lemeshow S, Saulnier F: A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 1993; 270:2957-2963
10. Lemeshow S, Teres D, Klar J, et al: Mortality probability models (MPM II) based on an international cohort of intensive care unit patients. JAMA 1993; 270:2478-2486
11. Knaus WA, Wagner DP, Draper EA, et al: The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991; 100:1619-1636
12. Richardson DK, Gray JE, McCormick MC, et al: Score for neonatal acute physiology: A physiologic severity index for neonatal intensive care. Pediatrics 1993; 91:617-623
13. Tarnow-Mordi W, Ogston S, Wilkinson AR, et al: Predicting death from initial disease severity in very low birth weight infants: A method for comparing the performance of neonatal units. BMJ 1990; 300:1611-1614
14. Knaus WA, Wagner DP, Lynn J: Short-term mortality predictions for critically ill hospitalized adults: Science and ethics. Science 1991; 254:389-394
15. Pollack MM, Ruttimann UE, Getson PR, et al: Accurate prediction of pediatric intensive care outcome: A new quantitative method. N Engl J Med 1987; 316:134-139
16. Pollack MM, Cuerdon TC, Getson PR: Pediatric intensive care units: Results of a national study. Crit Care Med 1993; 21:607-614
17. Fleiss JL: The Design and Analysis of Clinical Experiments. New York, John Wiley & Sons, 1986, pp 2-3
18. SAS/STAT User's Guide, Version 6. Fourth Edition. Volume 2. Cary, NC, SAS Institute, pp 1071-1126
19. Zhang P: Model selection via multifold cross validation. Ann Statistics 1993; 21:299-313
20. Hosmer DW, Lemeshow S: Applied Logistic Regression. New York, John Wiley & Sons, 1989
21. Ruttimann UE: Statistical approaches to development and validation of predictive instruments. Crit Care Clin 1994; 10:19-35
22. Flora JD: A method for comparing survival of burn patients to a standard survival curve. J Trauma 1978; 18:701-705
23. Hanley JA, McNeil BJ: A method of comparing the areas under the receiver operating characteristic curves derived from the same cases. Radiology 1983; 148:839-843
24. Kay R, Little S: Transformations of the explanatory variables in the logistic regression model for binary data. Biometrika 1987; 74:495-501
25. Pollack MM, Patel KM, Ruttimann UE, et al: Frequency of variable measurement in 16 pediatric intensive care units: Influence on accuracy and potential for bias in severity of illness assessment. Crit Care Med 1996; 24:74-77
26. Xavier C, Artigas A, Bion J, et al: A comparison of severity of illness scoring systems for intensive care unit patients: Results of a multicenter, multinational study. Crit Care Med 1995; 23:1327-1335
© Williams & Wilkins 1996. All Rights Reserved.
This article has been cited
Critical Care MedicineImprovement in Pediatric Critical Care OutcomesCritical Care Medicine
Critical Care MedicineImprovement in Pediatric Critical Care OutcomesCritical Care Medicine
Critical Care MedicinePerformance of the Pediatric Risk of Mortality ScoreCritical Care Medicine
Critical Care MedicineTo chest x-rays and beyond!!Critical Care Medicine
Critical Care MedicineCardiopulmonary resuscitation outcomes in childrenCritical Care Medicine
Critical Care MedicineLong-stay patients: Are there any long-term solutions? *Critical Care Medicine
Critical Care MedicineNoninvasive ventilation outside the intensive care unit: A new standard of care?*Critical Care Medicine
Critical Care MedicineQuality of care: National or international standards?Critical Care Medicine
Pediatric Critical Care MedicineRisk factors for central line-associated bloodstream infection in a pediatric cardiac intensive care unitPediatric Critical Care Medicine
Pediatric Critical Care MedicineRisk-adjusted intensive care unit outcomes and admission from our backyard*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineDetermining pediatric intensive care unit quality indicators for measuring pediatric intensive care unit safetyPediatric Critical Care Medicine
Pediatric Critical Care MedicineDo outcomes vary according to the source of admission to the pediatric intensive care unit?*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineZinc homeostasis in pediatric critical illness*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineCharacteristics and outcomes of interhospital transfers from level II to level I pediatric intensive care units*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineMulticenter cohort study of in-hospital pediatric cardiac arrest*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineMetabolic acidosis as an underlying mechanism of respiratory distress in children with severe acute asthma*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineEnergy expenditure in critically ill childrenPediatric Critical Care Medicine
Pediatric Critical Care MedicineRisk factors of acute renal failure in critically ill children: A prospective descriptive epidemiological study*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineNeuroendocrine dysfunction in pediatric critical illness*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineCollaborative Pediatric Critical Care Research Network: Looking back and moving forwardPediatric Critical Care Medicine
Pediatric Critical Care MedicineValidation of Pediatric Index of Mortality 2 (PIM2) in a single pediatric intensive care unit of Argentina*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineHave changes in ventilation practice improved outcome in children with acute lung injury?*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineDeveloping the Children’s Critical Illness Impact Scale: Capturing stories from children, parents, and staff*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineEvaluation of Pediatric Risk of Mortality (PRISM) scoring in African children with falciparum malariaPediatric Critical Care Medicine
Pediatric Critical Care MedicineSupport for PICUs and the PRISM scoring algorithmsPediatric Critical Care Medicine
Pediatric Critical Care MedicineNoninvasive positive pressure ventilation: Five years of experience in a pediatric intensive care unit*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineHyperglycemia is associated with morbidity in critically ill children with meningococcal sepsisPediatric Critical Care Medicine
Pediatric Critical Care MedicineElevation of brain natriuretic peptide levels in children with septic shock*Pediatric Critical Care Medicine
Pediatric Critical Care MedicineThe critically ill child with novel H1N1 influenza A: A case series*Pediatric Critical Care Medicine