Background
Application of scoring system in pediatric ICU (PICU) has a fundamental role in the evaluation of the risk for mortality, prediction of the patients’ outcome, and improving the quality of intensive care for the severely ill children 1.
The two frequently used mortality risk scoring systems in children are the Pediatric Risk of Mortality and the Pediatric Index of Mortality (PIM) scores 2,3. However, PIM is more simple to obtain data routinely from large numbers of sick children 4–7.
PIM was first introduced in 1997 by Shann et al.3 and was upgraded to PIM-2, with better predictability of the pediatric patients outcome 7.
PIM-2 score is applied in the first hour of intensive care admission, which is the least affected by any therapeutic interference. The one advantage of PIM over the Pediatric Risk of Mortality is the fact that the PIM-2 is based on 10 variables, all of which are collected at the point of admission, which facilitates data collection and avoids any influence on the results from 24 h of intensive management strategies 7.
Several studies have validated PIM-2 as an appropriate risk evaluation tool to predict mortality in pediatric patients from different localities, including developed and developing countries 3–8. However, information on its relevance in Egypt is lacking.
We aimed in this prospective study to assess the validity of PIM-2 in Middle Delta locality and the quality of critical care services in this unit and to compare it with the international reports.
Patients and methods
This study included 240 consecutive admissions to Tanta University Children’s Hospital at its PICU, Egypt, during the period from January 2014 to December 2014. Forty patients were excluded from the study because of PICU stay less than 2 h or transference from the PICU. There were 84 male and 116 female patients. Their ages ranged from 1 month to 15 years. Tanta PICU has 10 beds with seven ventilators.
Because of the location of the hospital, the PICU serves many governorates, including El-Gharbiya, El-Minofiya, El-Behaira, and Kafer El-Sheikh, with 17 million population.
The study was approved by the Ethics Committee of the Faculty of Medicine, Tanta University, (protocol number was 2040/09/13) and was conducted in accordance with the Helsinki Declaration. Written informed consent was obtained from each child’s guardian before any study procedure, with assurance of patient confidentiality.
All cases were subjected to thorough history taking and clinical examination on admission. All personal data, including patient name, age, sex, date of admission, date of discharge, length of stay, provisional diagnosis, and final diagnosis including systems, organs affected, and patient’s outcome (discharged or deceased) were recorded. PIM-2 7 was recorded immediately on admission (within first hour).
Patients’ outcome recorded either death or survival; the actual death rate and that expected using PIM-2 both were calculated and compared for the evaluation of Tanta PICU performance.
Definitions concerning the variables of PIM-2 score and the scoring method are according to PIM-2 developers’ guidelines 7,9. PIM-2 score was calculated using an online PIM-2 calculator 10. It is available online at http://www.openpediatrics.org or http://www.sfar.org/scores2/pim22 for predicting the risk for mortality that is based on the regression equation published in the PIM-2 scoring system manual 9.
Calculation of PIM-2 (and PIM-2 risk for death%):
;)
The PIM-2 score is calculated from various coefficients validated from research published by Slater et al.7. We used the first value of each variable measured within the period from the time of first contact to 1 h after arrival to the PICU. The first contact may be in Tanta University PICU, emergency department, or in hospital ward. If information is missing (e.g. base excess is not measured) zero was recorded, except for systolic blood pressure, which should be recorded as 120 9. We evaluated the performance of our unit using standardized mortality ratios (SMRs).
All PICU patients were evaluated daily using complete laboratory investigations, including complete blood count, blood gases, electrolyte level, blood sugar, and liver and kidney function tests.
The included patients were further classified on the basis of outcome as follows:
- The survivor group.
- The nonsurvivor group
On the basis of their PIM-2-predicted death rates, they were classified into five groups: less than 1, 1–5, 5–15, 15–30, and more than 30%.
Statistical analysis
The collected data were organized, tabulated, and statistically analyzed using Statistical Package for Social Studies, version 19 created using IBM (Illinois, Chicago, USA). Descriptive statistics were used as frequencies and percentages for categorical variables. For numerical values, the range, mean, and SDs were calculated. The median was used to summarize continuous variables that are not normally distributed. The Mann–Whitney test was used to test significant differences in the median of continuous variables such as the age, the weight, and the length of hospital stay between survivors and nonsurvivors. The unpaired Students t-test was used to compare two groups as regards quantitative data, whereas the χ2-test was used for comparison between two groups as regards qualitative data. The level of significance was adopted at P of 0.05 or less. The receiver operator characteristics curve was used to evaluate the capacity of PIM-2 score to discriminate between survivors and nonsurvivors; whenever its under-curve surface area is close to 1, the capacity of discrimination is considered to be high 11. The Hosmer–Lemeshow goodness-of-fit test was performed to calibrate PIM-2 score; acceptable calibration is evidenced at P value more than 0.05 12.
SMR (the ratio of the observed mortality to the expected mortality) was calculated for the whole population and for subgroups of patients using the OpenEpi statistical calculator online 10.
Results
Overall, during the 12 months of study period there were 200 patients, with an outcome of 126 survivors (50 male) and 74 nonsurvivors (34 male). The actual death rate was 37%.
Survivors’ ages ranged between 1 month and 15 years with a median of 21 months, and for nonsurvivors it ranged between 1.2 months and 14 years with a median of 5 months. There was a statistically significant decrease in weight in nonsurvivors compared with survivors; the median was 5 versus 10 kg, respectively. There was a significant increase in the length of PICU stay among nonsurvivors compared with survivors. The median was 8 versus 6 days in deceased patients and survivors, respectively. The maximum length of stay for survivors was 150 and 31 days for nonsurvivors.
PICU outcome in relation to diagnostic groups is listed in Table 1.
Table 1: Comparison of studied groups according to the system affected (diagnostic groups)
The highest rate of admission was seen in patients who presented with cardiac problems (19%), followed by respiratory (18%) and endocrine problems (18%). The worst prognosis was in patients who presented with either multiple organ dysfunction syndrome or hematological and oncological disorders (all died), followed by those with cardiac disorders (26 of 38 died), and finally those with renal disorders (eight of 14 died). The best prognosis was observed among patients suffering from either endocrine disorders, inborn errors of metabolism, or intoxication (all survived), followed by respiratory (28 of 36 survived), gastrointestinal and hepatic diseases (eight of 12 survived), neurological disorders (14 of 22 survived), and at last postoperative disorders (10 of 14 survived).
There was a statistically significant increase in PIM-2 mortality probability in nonsurvivors than in survivors, with a mean of 68.37±30.560 versus 13.8±14.44, respectively, as shown in Table 2.
Table 2: Comparison of studied groups according to the predicted mortality probability using the pediatric index of mortality-2 score (PIM-2 risk for death %)
Figure 1 presents screening power of PIM-2 score to predict mortality among patients admitted in the PICU. The area under the receiver operator characteristics was 0.763, indicating acceptable discriminatory power of the PIM-2 score, and hence the score had acceptable ability to distinguish between patients who survived and those who died.
Fig. 1: The area under the receiver operating characteristics curve for the Pediatric Index of Mortality-2 score.
Table 3 presents calibration of the PIM-2 score across all deciles of mortality risk. Calibration of PIM-2 score (which is the ability of the score to match the actual number of deaths) across all deciles of mortality risk was good according to the Hosmer–Lemeshow goodness-of-fit test. The differences between observed and expected number of deaths across the deciles of mortality risk were not significant (χ2=5.58, P=0.34) for the entire sample and for each single decile of mortality risk.
Table 3: Hosmer–Lemeshow goodness-of-fit analysis for the pediatric index of mortality-2 score across all deciles of mortality risk (calibration of the score across all deciles of mortality risk)
The incidence of expected deaths was 34% and that of observed death was 37%. The expected/observed ratio was 0.91; this means that PIM-2 score predicted 91% of actual deaths.
Table 4 shows that calibration of the score was good for all subgroups of patients according to the Hosmer–Lemeshow goodness-of-fit test (P>0.05, nonsignificant).
Table 4: Model fit and discrimination across all patients and different subgroups of patients
This means that the difference between the observed and score predicted mortality was not significant for all subgroups of patients. However, the score overpredicts deaths in some patient subgroups with SMR less than 1, and underpredicts deaths in other subgroups with SMR more than 1. However, the difference between observed and expected deaths is still not significant.
Discussion
The PICU aims to introduce a highly qualified care with getting the best results and better progress for critically ill children 13.
In the background of intensive care management, a basis and objective way to define and quantify severity of illness is through the development of probability models of predicting mortality risk 14,15. Hence, this score will help to track our own quality and efficiency of care progress (internal benchmarking); this was the second objective. The third objective was to compare our performance with that of other PICUs (external benchmarking).
For the present study we chose the PIM-2 score because it is one of the most applicable severity score published for children. It has a free algorithm to calculate mortality risk, whereas other scores require a license, and the small number of variables (10) makes it very simple to collect. In addition, the Australian and New Zealand Pediatric Intensive Care network found that the PIM-2 score performed better than other rating systems 5.
The present study showed an actual mortality rate of 37%, which is considerably high. Similar results were reported in studies in developing countries: 24.3, 46.2, and 28% in studies conducted in India 6,15,16 and 28.7% in Pakistan 17.
This high death rate in studies from developing countries may be attributed to admission of hopeless cases, late referral to the PICU, or limited resources 15.
In contrast, studies from other countries showed low death rates: Italy, 8.5% and 4.4% 18,19; Japan, 5.2% 20; and Brazil, 14.13% 21.
The present study showed that there was a significant statistical increase in the PIM-2 score in nonsurvivors than in survivors, with the mean PIM-2 score of 68.37±30.56 versus 13.8±14.44, respectively. This means that a higher PIM-2 score was significantly associated with an increased risk for nonsurvival. This is in accordance with the findings of Gandhi et al.15, who found a statistical significance in the association of the predicted death rate with mortality with a mean predicted death rate of 96.14±19.98 versus 88.87±20.05 in nonsurvivors and survivors, respectively.
As regards the validity of the score in Tanta University PICU, discrimination and calibration are two very important characteristics of a score that were used to evaluate its statistical performance, and hence both are used to judge its validity in our PICU.
The present study showed that the area under the curve (AUC) for PIM-2 was 0.763. This indicated that PIM-2 had an accepted AUC, and hence accepted discriminatory power.
Several studies reported the AUC for PIM-2. In Fayoum University hospital in Egypt, AUC was 0.75 22. In Cairo University hospital it was 0.796 23.
Studies in developing countries such as in India 6,15, Iran 24, Pakistan 17, Barbados 25, and Argentina 26 showed results comparable to the present study with an AUC of 0.851, 0.843, 0.795, 0.88, 0.82, and 0.84, respectively – that is, they had acceptable discrimination.
Most of the studies in developed countries showed good or excellent AUC of at least 0.9 and discrimination, with 0.9, 0.79, 0.92, 0.889, and 0.871 in Australia, New Zealand, UK 7, Italy 18, Japan 20, Hong Kong 27, and Spain 28, respectively.
In the present study, the specificity of PIM-2 score was relatively not high (75.7%). This is in accordance with the findings of Gandhi et al.15, who found a low specificity (65.6%) of PIM-2 score. They stated that the low specificity of PIM-2 score denotes that not all patients with high predicted death rate may die. This reflects that effective interventions in the PICU reduce mortality of those who have high predicted death rate at the time of admission, and thereby indicated good performance of the PICU.
The validity of the present study using the Hosmer–Lemeshow goodness-of-fit test showed adequately fit calibration for PIM-2 (χ2=5.58) (P=0.34).
The results were as follows: Fayoum 22, χ2=1.410, d.f.=8, P=0.9; Cairo 23, χ2=2.850, d.f.=8, P=0.943); Pakistan 17, χ2=9.65, P=0.29; Iran 24, χ2=5.161, P=0.7; Trinidad 29, χ2=5.61, d.f.=8, P=0.69; Barbados 25, χ2=5.64, d.f.=7, P=0.58; Australia 7, χ2=11.5, P=0.17; Canada 30, P=0.17; Japan 20, χ2=4.8, P=0.44; Italy 18, χ2=9.86, P=0.26, P=0.48; Spain 28, χ2=4.87, d.f.=8, P=0.85; and Brazil 21, χ2=12.26, P=0.1396. This shows that all these studies had good calibration of PIM-2 score.
However, in a study in Argentina 26, an inadequate calibration was observed and significant differences (χ2:71.02, d.f.=8, P<0.001) were described for the general population and in most of the mortality risk deciles. The lack of calibration of the PIM2 score in their studied population in which respiratory disorders constitute 29% had a more severe status.
However, in the study by Atti et al. 18, who evaluated the performance of PIM-2 in cardiac and mixed ICUs in a tertiary children’s referral hospital in Italy, the Hosmer–Lemeshow test showed a suboptimal overall goodness-of-fit (P<0.001). This result was mainly due to the overprediction of deaths in the highest risk group (114.7 vs. 53; P<0.001).
The observed number of deaths in the present study was 74 and deaths predicted using PIM-2 were 68. Thus, SMR (observed deaths/expected deaths) estimated using PIM-2 was 1.08 at 95% confidence interval (CI) of 1.05–1.1, indicating underestimation of death using PIM-2.
The present study is in accordance with similar studies in other localities within Egypt such as in Fayoum 22, which reported an SMR of 1.55 and the study in Cairo 23, which reported an SMR of 1.92. Developing countries have reported an overall observed mortality that is higher than the PIM-2-predicted mortality, with a subsequent SMR of more than one. In Trinidad 29, the predicted mortality using the PIM-2 in children was 34.8% with the observed mortality rate being 30%. In a study conducted in Iran 24, the number of deaths observed was 36 (15%) and the expected mortality using PIM-2 was 20 (8.3%); SMR was 1.8 (95% CI: 1.28–2.46). In a study conducted in Pakistan 17, SMR was 1.4 for PIM-2. In a study conducted in Barbados 25 the mean predicted mortality was 6.2% (95% CI: 4.3–8.1) and the observed mortality rate was 5.5%, with an SMR of 0.89.
In contrast, the validity of PIM-2 in PICU settings in other countries reported an observed mortality lower than the PIM-2 predicted mortality and thus a SMR less than 1 – that is, overestimation of death. In Italy 18 SMR was 0.7 (95% CI: 0.6–0.8) and in Japan SMR was 0.77 20.
The case-mix of the studied patients as regards the diagnostic groups differ between the original study by Slater et al.7 and the present study as follows: cardiac cases, 25.5 versus 19%; respiratory cases, 21.5 versus 18%; neurological cases, 9.3 versus 11%; postoperative cases, 19.2 versus 7%; miscellaneous cases, 24.5 versus 45%.
In the original study by Slater et al.7, the observed deaths equal the expected deaths with an SMR of 1.
In the study by Sankar et al. in India 16, they found SMR to be equal to 1. They presume that factors like the threshold for initiating and discontinuing support, timing of intensive care admissions, and quality of care as well as the accuracy of data collection might have contributed to the near-perfect SMR in their unit.
It can be concluded that PIM-2 can be used as a good prediction model for pediatric mortality and as a tool for assessing the overall quality of care in our PICU. PIM-2 is easily calculated and is freely available, and thus provides a good reason for ICU setting for admission of high-risk patients in the light of the limited PICU bed complement capacity in relation to the demands. The higher observed death rate compared with the PIM-2 predicted death rate (underprediction of mortality by PIM-2) may be due to differences in the population under study compared with the original population in which PIM-2 was first investigated and developed, inadequate resources, and small number of included participants that may be a limitation. Patient’s mortality is not only affected by ICU performance but also depends on many other factors such as demographic and clinical characteristics of population, infrastructure and nonmedical factors (management and organization), case mix, and admission practice. As the PIM-2 score can be determined at the earliest part of patient management, this will be very useful in counseling parents. Further larger scale studies in cooperation with other universities of Egypt as well as neighboring countries are required for the optimal use of the score within our region.
Table: No title available.
Acknowledgements
Conflicts of interest
There are no conflicts of interest.
References
1. Marcin JP, Pollack MM. Review of the acuity scoring systems for the pediatric intensive care unit and their use in quality improvement. J Intensive Care Med 2007; 22:131–140.
2. Pollack MM, Ruttimann UE, Getson PR. Pediatric risk of mortality (PRISM) score. Crit Care Med 1988; 16:1110–1116.
3. Shann F, Pearson G, Slater A, Wilkinson K. Pediatric index of mortality (PIM): a mortality prediction model for children in intensive care. Intensive Care Med 1997; 23:201–207.
4. Taori RN, Lahiri KR, Tullu MS. Performance of PRISM (Pediatric Risk of Mortality) score and PIM (Pediatric Index of Mortality) score in a tertiary care pediatric ICU. Indian J Pediatr 2010; 77:267–271.
5. Slater A, Shann F. ANZICS Paediatric Study GroupThe suitability of the pediatric index of mortality (PIM), PIM2, the pediatric risk of mortality (PRISM), and PRISM III for monitoring the quality of pediatric intensive care in Australia and New Zealand. Pediatr Crit Care Med 2004; 5:447–454.
6. Thukral A, Lodha R, Irshad M, Arora NK. Performance of pediatric risk of mortality (PRISM), pediatric index of mortality (PIM), and PIM2 in a pediatric intensive care unit in a developing country. Pediatr Crit Care Med 2006; 7:356–361.
7. Slater A, Shann F, Pearson G. Pediatric Index of Mortality (PIM) Study Group, PIM2: a revised version of the Pediatric Index of Mortality. Intensive Care Med 2003; 29:278–285.
8. Jones GD, Thorburn K, Tigg A, Murdoch IA. Preliminary data: PIM vs. PRISM in infant and children cardiac surgery in a UK PICU. Intensive Care Med 2000; 26:145–153.
9. PIM2 & PIM3 for the ANZPIC Registry Information Booklet version August 2014.
10. OpenEpiStandardized mortality ratio and 95 confidence interval. Available at:
http://www.openepi.com/SMR/SMR.htm.
11. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29–36.
12. Lemeshow S, Hosmer DW Jr. A review of goodness of fit statistics for use in the development of logistic regression models. Am J Epidemiol 1982; 115:92–106.
13. Choi KM, Ng DK, Wong SF, Kwork KL, Chow PY, Chan CH, Ho JC. Assessment of the Pediatric Index of Mortality (PIM) and the Pediatric Risk of Mortality (PRISM III) score for the prediction of mortality in a pediatric intensive care unit in Hong Kong. Hong Kong Med J 2005; 11:97–103.
14. Tilford JM, Simpson PM, Green JW, Lensing S, Fiser DH. Volume-outcome relationships in pediatric intensive care units. Pediatrics 2000; 106 (Pt 1):289–294.
15. Gandhi J, Sangareddi S, Varadarajan P, Suresh S. Pediatric index of mortality 2 score as an outcome predictor in pediatric Intensive Care Unit in India. Indian J Crit Med 2013; 17:288–291.
16. Sankar J, Singh A, Sankar MJ, Joghee S, Dewangan S, Dubey N. Pediatric Index of Mortality and PIM2 scores have good calibration in a large cohort of children from a developing country. BioMed Res Int 2014; 2014:907871.
17. Qureshi AU, Ali AS, Ahmad TM. Comparison of three prognostic scores (PRISM, PELOD and PIM2) at pediatric intensive care unit under Pakistani circumstances. J Ayub Med Coll Abbottabad 2007; 19:49–53.
18. Atti M, Cuttini M, Rava L, Rinaldi S, Brusco C, Cogo P, et al. Performance of the pediatric Index of Mortality 2(PIM-2) in cardiac and mixed intensive care units in a tertiary children’s referral hospital in Italy. BMC Pediatr 2013; 13:100.
19. Wolfler A, Silvani P, Musicco M, Salvo I. Italian Pediatric Sepsis Study (SISPe) Group. Pediatric Index of Mortality 2 in Italy: a multicenter, prospective, observational study. Intensive Care Med 2007; 33:1407–1413.
20. Imamura T, Nakagawa S, Goldman R, Fujiwara T. Validation of pediatric index of mortality 2 (PIM2) in a single pediatric intensive care unit in Japan. Intensive Care Med 2012; 38:649–654.
21. Netto AL, Muniz VM, Zandonade E, Maciel EL, Bortolozzo RN, Costa NF, LimongiRda S. Performance of the Pediatric Index of Mortality 2 in a pediatric intensive care unit. Rev Bras Ter Intensiva 2014; 26:44–50.
22. El-S BO, Algameel AA, Eldash HH. Application of pediatric index of mortality version 2 score in pediatric intensive care unit in an African developing country. Pan Afr Med J 2014; 17:185.
23. Youssef MRL, Mosleh H, Labib JR. Assessment of the performance of the Pediatric Index of Mortality 2 (PIM2) among Egyptian pediatric patients admitted to the intensive care. Egypt Pediatr Assoc Gaz 2014; 62:65–71.
24. Salamati P, Talaee S, Eghbalkhah A, Chaman R, Mokhtari Z, Azarshahin M. Validation of pediatric index of mortality-2 scoring system in a single pediatric intensive care unit in Iran. Iran J Pediatr 2012; 22:481–486.
25. Hariharan S, Krishnamurthy K, Grannum D. Validation of Pediatric Index of Mortality-2 scoring system in a pediatric intensive care unit, Barbados. J Trop Pediatr 2011; 57:9–13.
26. Fernández AL, Arias López MP, Ratto ME, Saligari L, Siaba Serrate A, de la Rosa M, et al. Validation of the Pediatric Index of Mortality 2 (PIM2) in Argentina: a prospective, multicenter, observational study. Arch Argent Pediatr 2015; 113:221–228.
27. Ng DK, Miu TY, Chiu WK, Hiu NT, Chan CH. Validation of pediatric index of mortality 2 in three pediatric intensive care units in Hong Kong. Indian J Pediatr 2011; 78:1491–1494.
28. Prieto Espuñes S, López-Herce Cid J, Rey Galán C, Medina Villanueva A, Concha Torre A, Martínez Camblor P. Prognostic indexes of mortality in pediatric intensive care units. An Pediatr (Barc) 2007; 66:345–350.
29. Hariharan S, Merritt-Charles L, Chen D. Risk-adjusted outcome evaluation in a multidisciplinary intensive care unit. West Indian Med J 2007; 56:240–245.
30. Lacroix J, Cotting J. Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network Severity of illness and organ dysfunction scoring in children. Pediatr Crit Care Med 2005; 6 (Suppl):S126.
