Journal Logo

Methods and Instruments

Reliability and Validity of a Pediatric Fall Risk Assessment Scale for Hospitalized Patients in Taiwan

Chang, Ching-Mei PhD, RN; Wen, Cheng-Fan MPH; Lin, Hsien-Feng MD

Author Information
Quality Management in Health Care: April/June 2021 - Volume 30 - Issue 2 - p 121-126
doi: 10.1097/QMH.0000000000000305
  • Open

Abstract

Falls are the most commonly reported adverse hospital events.1–3 Falls are a serious problem associated with morbidity, mortality, and prolonged hospitalizations in adult and elderly inpatients. However, little focus has been placed on pediatric inpatient falls. Hospitalized patient falls are defined as “unintentionally coming to rest on the ground, floor, or other lower level, with or without injury”; fall events are medical errors that have become important indicators of health care quality.2,4 In the United States, pediatric fall rates range from 0.4 to 3.8 per 1000 patient-days.5–7 In Taiwan, pediatric fall rates in hospitals range from 0.04 to 0.1 per 100 patient-days.8

Studies show that factors related to falls in pediatric patients include child human factors, environmental human factors, biomechanical factors, caregiver human factors, and system factors. Some studies have reported that a majority of pediatric falls are related to environmental risks (eg, beds, cribs, and chairs); approximately half of the patients in those studies fell from their cribs after climbing over the side rails or fell off from their height-adjustable beds after climbing over the bed rail.4,6,9, Many studies have reported the characteristics of pediatric patients with a fall, including those who fall in the presence of their parents.1,10 The reason that most pediatric falls occur in the presence of parents may be because children feel more comfortable when a family member is present and thus tend to be more active (eg, running, climbing, jumping).9,11 A majority of the falls occur among boys, in the age group from 1 to 5 years, with normal mobility status and with no history of previous falls.12 Pediatric fall-related injury rates have been reported to be 36% to 58% in related studies.10,13 Fall-related injuries include fractures and head injuries. Minor falls may lead to skin bruising, whereas severe falls may lead to fracture and extended hospital stays.2,14

How can we prevent pediatric fall occurrence? Some studies indicated the most important fall prevention strategy is to assess whether patients are at risk of falling using highly predictive fall risk assessment tools (FRATs).5,7,15 Thus, FRATs are a very effective intervention strategy for maintaining and improving fall prevention programs. But most FRATs focus on elderly people and risk for intrinsic-type falls (ie, diagnosis, medications, fluid imbalance, physical impairments). Extrinsic falls (nonphysiologic falls) and additional factors related to pediatric falls include horseplay (children running or climbing on the furniture or equipment) and little or lack of focus. Some systematic reviews of the literature show pediatric FRATs that have commonly been used include the GRAF-PIF scale, Humpty Dumpty scale, and CHAMPS pediatric fall risk assessment tool used worldwide.5,7,16–21 The most common items included patient diagnoses, use of sedative medications, and mobility. The GRAF-PIF scale is a 5-item scale and has sensitivity of 0.75 and specificity of 0.76.10 The Humpty Dumpty scale is a 7-item scale, with items including age, gender, diagnosis, cognitive impairments, environmental factors, response to surgery sedation anesthesia, and medication usage. The pooled sensitivity and specificity of the 9 studies were 0.85 and 0.24, respectively.7,14 The CHAMPS scale is a 4-item scale including physical disability, age, cognitive impairment, and previous fall; its sensitivity and specificity are not reported.

Most of these pediatric FRATs have been developed in the United States. The body of literature on fall prevention tools and prevention programs in pediatrics is growing slowly.7,20 There is no pediatric fall assessment scale (PFAS) developed in Asian countries. In Taiwan, most hospitals or institutions use adult FRATs or use scales developed internally. Only a few institutions use a validated tool. In summary, pediatric FRATs and prevention protocols in hospitals have not been standardized.8 Studies have indicated that adult FRATs are often borrowed to create tools for pediatric patients. But adult-related fall tools poorly predict the risk of pediatric falls. The variable definitions associated with these tools are not often applicable to children.9,20 Pediatric fall assessment tools developed in the United States have several flaws due to differences in culture, hospital factors, and health care in Taiwan. Thus, developing a PFAS with reliability and validity for Asian countries is an important task. Together with the aforementioned facts, further research and development of fall risk assessment tools and evidence-based practice interventions are needed. Therefore, this study aimed to investigate the reliability and validity of a PFAS to help prevent falls in hospitalized pediatric patients in Taiwan.

METHODS

Study design

This is a retrospective study that was conducted in a 130-bed pediatric teaching hospital in central Taiwan that admits approximately 29 954 pediatric inpatients per year and includes the pediatric ward, neonatal intensive care unit, pediatric intensive care unit, and sick baby room. The average hospital stay at the time was 5 to 6 days. This study was conducted from 2011 to 2013.

Subjects

Subjects were recruited through a teaching hospital in central Taiwan. The inclusion criteria were patients (1) younger than 18 years and (2) who had experienced hospitalization in the pediatric ward. Exclusion criteria were (1) patients not admitted to pediatric wards and (2) outpatient patients.

Survey methods

The PFAS was developed by review of the literature, data collection on the causes of falls in hospitalized children over the years, and experiences of experts in pediatric medicine including a clinical pediatrician, a senior pediatric nurse supervisor, and a pediatric nursing teacher. This study combined the nursing information system (NIS) and the electronic incident reporting system (EIRS). The EIRS is an electronic reporting system that allows clinicians, health care workers, and health care managers to record and manage adverse clinical incidents that occur in hospitals. When subjects were admitted to the pediatric ward, nurses reported their situation daily by implementing PFAS assessment and registration. When an adverse fall event occurred, the nurses registered it into the EIRS. The data were extracted from daily fall assessment and incident reports of the NIS and the EIRS.

Instruments

The PFAS is a self-developed scale, with the item development steps composed of (1) identification of the domain(s) and item generation, (2) consideration of content validity, (3) pretesting questions, (4) sampling and survey administration, (5) item reduction, (6) extraction of latent factors, (7) tests of dimensions, (8) tests of reliability, and (9) tests of validity. Based on these steps, the PFAS includes 7 risk factors: age (1-4 points), sex (1-2 points), disease factors (1-3 points), history of falling (1-3 points), environmental factors (1-2 points), anesthesia/sedation (1-3 points), and medication use (1-3 points). The total score ranges from 7 to 20 points. Both low-risk and high-risk points of the PFAS are set on the basis of the area under the receiver operating characteristic (ROC) curve (AUC-ROC). Patients with a total score of 11 points or less were classified as the low-risk group and those with a score between 12 and 20 points were classified as the high-risk group (Table 1).

Table 1. - Pediatric Fall Assessment Scale
Parameter Criteria Score
Age <2 y 4
2 to <5 y 3
5 to <12 y 2
≥12 y 1
Gender Male 2
Female 1
Diagnosis Neurological disease 3
Behavioral disease 2
Other disease 1
Fall history ≥3 falls over the past 3 mo 3
1-2 falls over the past 3 mo 2
No falls 1
Environmental Factors Using aids (eg, IV line, wheelchair and IV stand) 2
Not using aids (eg, IV line, wheelchair, and IV stand) 1
Surgery/sedation/anesthesia <24 h
3
24-48 h 2
≥48 h or none 1
Medication use (sedative, barbiturates, hypnotics, phenothiazine, laxatives, diuretics, hypoglycemic agents, antihistamine) ≥2 of the meds 3
One of the meds 2
None of the meds 1
Highest score 20
Lowest score 7
High-risk group ≥12
Low-risk group ≤11
Abbreviation: IV, intravenous.

Statistical analysis

The association between risk factors and falls was analyzed to determine whether known risk factors had an impact on the incidence of falls (SPSS version 13.0; IBM Corporation, Armonk, New York). The frequency and percentage of falls and the general characteristics of the patients were analyzed using descriptive statistics. Internal consistency was assessed with the use of Cronbach's α. Test-retest reliability was assessed with the use of the first and second measurements. The relative risk of falls and 95% confidence intervals (CIs) were analyzed on the basis of patients' general characteristics. A P value of less than .001 was considered statistically significant. Predictive validity was estimated by measuring the sensitivity, specificity, ROC curve, and AUC-ROC. The AUC-ROC and sensitivity and specificity values were then analyzed to determine the validity of the PFAS.

Ethical considerations

This study adhered to all relevant ethical research criteria and was approved by the institutional review board of the target hospital.

RESULTS

Response and patient demographics

The data of 2126 patients were analyzed. Most of the patients were younger than 2 years (1093 patients, 51.4%), were male (1205 patients, 56.7%), 94.8% had no history of falls, were using aids (eg, intravenous line, wheelchair) during hospitalization (74.5%), and had never used medication(s) associated with an increased risk of falling (76.5%) (Table 2). Among these, 110 children had a fall. Through PFAS screening, 1535 of 2126 patients belonged to the low-risk group and 591 were in the high-risk group; 71.8% of the 110 falls were reported in the high-risk group (Table 3).

Table 2. - Characteristics of Participants (N = 2126)
Items Categories n %
Age ≥12 y 166 7.8
5 to <12 y 370 17.4
2 to <5 y 497 23.4
<2 y 1093 51.4
Gender Female 921 43.3
Male 1205 56.7
Diagnosis Behavioral disease 37 1.7
Neurological disease 39 1.8
Other disease 2050 96.4
Fall history No 2016 94.8
Yes (1-2 falls over the past 3 mo) 237 5.1
>3 falls over the past 3 mo 97 4.6
Environmental factors Not using aids (eg, IV drip and wheelchair) 542 25.5
Using aids (eg, IV drip and wheelchair) 1584 74.5
Surgery/sedation/anesthesia ≥48 h or none 2024 95.2
24-48 h 52 2.4
<24 h 50 2.4
Medication use None of the meds 1627 76.5
One of the meds 499 23.5
≥2 of the meds 0 0.0
Abbreviation: IV, intravenous.

Table 3. - Percentages of the High-Risk and Low-Risk Groups (Children Who Fell and Those Who Did Not Fall) (N = 2126)
Fall Total Pearsonʼs Chi-Squared Test
No Yes
Low risk, n (%) 1504 (74.6) 31 (28.2) 1535 (72.2) .000a
High risk, n (%) 512 (25.4) 79 (71.8) 591 (27.8)
Total, n (%) 2016 (100.0) 110 (100.0) 2126 (100.0)
aP ≤ .001.

Reliability and validity of the PFAS

Internal consistency of the PFAS was determined using Cronbach's α, which was 0.71. After 24 hours, test-retest reliability was 0.89, which was determined by retesting age, sex, disease factors, history of falls, environmental factors, anesthesia/sedation, and medication use of the PFAS. The Pearson correction coefficient was 0.89 (P = .000). Based on the content validity index, validity of the PFAS was 0.86. The investigators then assessed the average score of each patient in this study and the standard score, which distinguishes a patient as being at a high or low risk for falls. These participants were then evaluated whether they had fallen. Based on the average scores, for the sensitivity was 74.6% and the specificity was 71.8%; Pearson's chi-squared test was P = .000 (Table 3). When the minimum score for a high risk of falls was 11.5 points, sensitivity was 71.8% and specificity was 74.6%. The risk factors in the model showed a statistically significant relationship with patient falls (odds ratio [OR] = 7.486; 95% CI, 4.883-11.477; P =.000). Regarding the overall validity of the PFAS, the AUC-ROC was 0.79 (Table 4; Figure).

Figure.
Figure.:
Receiver operating characteristics curves.
Table 4. - Receiver Operating Characteristics Curves
Area Standard Error Significance 95% Confidence Interval
Lower Upper
7.486 .026 .000a 4.883 11.477
aP ≤ .001.

Comparison of the PFAS and previously existing fall risk assessment tools

Among the patients in the present study, 110 (5.2%) had fallen, most of whom were younger than 2 years (44/110, 40%), male (72.7%), using aids (eg, intravenous line, wheelchair) during hospitalization (93.6%), and not using relevant medication when they had a fall (65.5%). Regarding other diseases diagnosed, most of the patients had respiratory diseases (76.4%), followed by gastrointestinal (22.7%) and urinary tract diseases (2.7%). The prevalence of falls was 5.2% (Table 5). When the 110 fall incidents were compared between the PFAS and the previously existing fall risk assessment tool, the PFAS identified 89.1% of those who eventually fell as being at high risk, which was much higher than that reported by the previous scale (4.7%). For ethnic patients younger than 5 years, the PFAS identified 98.7% of those in the high-risk groups, which was much higher than that reported by the previous scale (4.6%) (P = .000, P < .0001). The data show that PFAS score was much higher than that reported by the previous scale screening the high-risk fall groups and was statistically significant (Table 6).

Table 5. - Characteristics of Falls (N = 110)
Items n %
Age
≥12 y 11 10.0
5 to <12 y 25 22.7
2 to <5 y 30 27.3
<2 y 44 40.0
Gender
Female 30 27.3
Male 80 72.7
Diagnosis
Other disease 99 90.0
Behavioral disease 6 5.5
Neurological disease 5 4.5
Environmental factors
Using aids 7 6.4
Not using aids 103 93.6
Surgery/sedation/anesthesia
≥48 h or none 100 90.9
24-48 h 6 5.5
<24 h 4 3.6
Medication use
None of the meds 72 65.5
One of the meds 38 34.5
Prevalence of falls 110/2126 5.2

Table 6. - Comparison of Old and New Forms of Fall Risk Scale (N = 110)
Age Old Scale PFAS P
Non-High Risk High Risk Non-High Risk High Risk
≥12 y 11 (100%) 0 (0.0%) 8 (72.7%) 3 (27.3%)
5 to <12 y 23 (92.0%) 2 (8.0%) 10 (40.0%) 15 (60.0%)
<5 y 71 (95.4%) 3 (4.6%) 1 (1.3%) 73 (98.7%) .000a
Total 105 (95.3%) 5 (4.7%) 12 (10.9%) 98 (89.1%)
Abbreviation: PFAS, pediatric fall assessment scale.
aP ≤ .001.

DISCUSSION

A fall is one of the most common adverse events that occur in hospitals.1–3 Therefore, it is very important to identify patients with a high risk of falling to prevent falls. Fall risk assessment scales have been developed to identify patients at a high risk of falling; however, it is necessary to identify the most easily applicable, appropriate, and valid fall risk assessment scale for hospitalized patients in Taiwan.

Internal consistency analysis showed a high reliability for the PFAS. Furthermore, test-retest reliability was measured as an additional method of assessing reliability, considering that the correlation coefficient in the range of 0.50 to 0.75 generally represents moderate to high reliability and a correlation coefficient of more than 0.75 indicates very high reliability. However, this studyʼs result shows the test-retest reliability was 0.89. These data represent a high degree of reliability for the internal consistency of the PFAS.

An assessment scale with a higher sensitivity is preferred so that potential fall cases are not missed. Sensitivity is the ratio of people who are expected to fall according to the scores of patients with a history of falls, while specificity is the ratio of people who are not expected to fall according to the score of patients who do not have a history of falls. According to some studies, the AUC-ROC analysis shows a relationship between sensitivity and specificity according to the continuous change of cutoff scores.1 Cutoff scores represent a varying degree of risk. They are generally selected on the basis of the inflection point where those who had a fall and those without a fall are identified with similar accuracy, balancing sensitivity and specificity. In this study, 11.5 points was chosen as the minimum score for a high risk of falls, with sensitivity of 71.8% and specificity of 74.6%. The risk factors in the model had a statistically significant relationship with patient falls (OR = 7.486; 95% CI, 4.883-11.477; P = .000).

The AUC-ROC was used to assess overall validity of the PFAS, which was 0.79. Theoretically, 0.5 < AUC < 1. Its value is higher, indicating a higher diagnosis value of the test. When the AUC is close to 0.5, the ROC curve is close to the diagonal and the diagnostic test loses clinical significance. The AUC-ROC of less than 0.70 indicates a low accuracy of diagnosis; between 0.70 and 0.90, diagnostic accuracy is medium; and above 0.90, diagnostic accuracy is high. For this article, the AUC-ROC = 0.79, showing a moderate diagnosis accuracy. It shows appropriate decisive power to assess fall risk. The PFAS is a 7-item instrument, with total scores ranging from 7 to 20 points. Patients scoring 12 or more points are classified as being at high risk for falls. Both sensitivity and specificity of the PFAS were 71.8% and 74.6%, respectively, indicating strong significance. In comparison, the Humpty Dumpty Falls Scale (HDFS), which also has 7 items, has been shown to have sensitivity of 85% and specificity of 24%. The GRAF-PIF scale, which has 5 items, has been shown to have sensitivity of 75% and specificity of 76%.5,19 The items on the HDFS include age, gender, diagnosis, cognitive impairments, environmental factors, response to surgery and anesthesia/sedation, and medication use.14,19 The PFAS differs in that it does not include any items for the assessment of cognitive impairments, but similar to the HDFS, it does have items for the history of falls and environmental factors. Nevertheless, the scoring criteria and point totals are not the same because of the hospital factor and culture differences.

The GRAF reported that the diagnosis of respiratory/pulmonary and neurological (seizure) conditions was associated with an increased incidence of falls.10 These findings are consistent with those reported in the present study, that is, more patients with a history of falls had been diagnosed with respiratory disease. The results of the present study show that falls frequently occurred in male pediatric patients (72.7%) and those 2 years or younger, which is consistent with the findings of Cooper and Nolt.16 In addition, most patients who had a fall were using aids and most pediatric falls resulted from climbing over the bed rails; therefore, falling out of bed was identified as a child's activity associated with an increased risk of falls. These results are consistent with those of other studies.4,6,16

However, it is reasonable to believe that the use of a fall risk assessment tool heightens awareness about fall risk and fall prevention strategies to mitigate the risk. Based on the knowledge and experience of the research team, it is clear that children have additional inherent risks for falls that would be extremely hard to capture by an objective tool like the 2 studied. There is still much scope to improve existing tools and be creative in how we can help prevent pediatric falls. The development and ongoing study and refinement of validated fall risk assessment tools are time-consuming and compete with other patient safety initiatives. Clinicians and researchers across the globe continue to work together to share best practices and refine pediatric fall prevention tools.

CONCLUSIONS

In the present study, the PFAS was shown to be an appropriate scale for assessing the fall risk of hospitalized pediatric patients in Taiwan. The PFAS is expected to give health care providers a point of reference for assessing fall risks in pediatric patients. The findings also suggest that children younger than 2 years may be at the highest risk of falling and should therefore be closely monitored.

Relevance to clinical practice

To efficiently predict the occurrence of falls in pediatric patients admitted to acute care hospitals.

Research limitations

This studyʼs research limitations include collection of data in a single inpatient pediatric unit and that most of the patients were younger than 2 years (51.4%), therefore affecting the average age of the population under study. Besides that, this was a retrospective study design that only enrolled patients who had fallen during hospitalization in a certain teaching hospital in central Taiwan. This study used hospitalized children of the same group as the sample of retest reliability, and considering the days hospitalized and changes of diseases. Thus, 24 hours is used for test-retest reliability. Therefore, the generalizability of the research results is limited.

REFERENCES

1. Kim KS, Kim JA, Choi YK, et al. A comparative study on the validity of fall risk assessment scales in Korean hospitals. Asian Nurs Res (Korean Soc Nurs Sci). 2011;5(1):28–37. doi:10.1016/S1976-1317(11)60011-X.
2. Chen XL, Liu YH, Chan DKY, Shen Q, Van Nguyen H. Characteristics associated with falls among the elderly within aged care wards in a tertiary hospital: a retrospective. Chin Med J (Engl). 2010;123(13):1668–1672.
3. Mion LC, Chandler AM, Waters TM, et al. Is it possible to identify risks for injurious falls in hospitalized patients? Jt Comm J Qual Patient Saf. 2012;38(9):408–413. doi:10.1016/s1553-7250(12)38052-5.
4. Ryan-Wenger NA, Dufek JS. An interdisciplinary momentary confluence of events model to explain, minimize, and prevent pediatric patient falls and fall-related injuries. J Spec Pediatr Nurs. 2013;18(1):4–12. doi:10.1111/jspn.12009.
5. Ryan-Wenger NA, Kimchi-Woods J, Erbaugh MA, LaFollette L, Lathrop J. Challenges and conundrums in the validation of pediatric fall risk assessment tools. Pediatr Nurs. 2012;38(3):159–167.
6. Schaffer PL, Daraiseh NM, Daum L, Mendez E, Lin L, Huth MM. Pediatric inpatient falls and injuries: a descriptive analysis of risk factors. J Spec Pediatr Nurs. 2012;17(1):10–18. doi:10.1111/j.1744-6155.2011.00315.x.
7. McNeely HL, Thomason KK, Tong S. Pediatric fall risk assessment tool comparison and validation study. J Pediatr Nurs. 2018;S0882-5963(17):30436–30440.
8. Taiwan Joint Commission on Hospital Accreditation. Taiwan patient-safety reporting system annual report 2014. http://www.tpr.org.tw/images/pic/files. Accessed January 12, 2015.
9. Razmus I, Davis D. The epidemiology of falls in hospitalized children. Pediatr Nurs. 2012;38(1):31–35. doi:10.1016/j.jpedsurg.2017.04.014.
10. Graf E. Magnet childrenʼs hospitals: leading knowledge development and quality standards for inpatient pediatric fall prevention programs. J Pediatr Nurs. 2011;26(2):122–127. doi:10.1016/j.pedn.2010.12.007.
11. Angalakuditi MV, Angalakuditi N. Causes and consequences of injuries in children in Western Australia. Risk Manag Healthc Policy. 2011;4:107–111. doi:10.2147/RMHP.S23226.
12. AlSowailmi BA, AlAkeely MH, AlJutaily HI, Alhasoon MA, Omair A, AlKhalaf HA. Prevalence of fall injuries and risk factors for fall among hospitalized children in a specialized childrenʼs hospital in Saudi Arabia. Ann Saudi Med. 2018;38(3):225–229. doi:10.5144/0256-4947.2018.225.
13. Kingston F, Bryant T, Speer K. Pediatric falls benchmarking collaborative. J Nurs Adm. 2010;40(6):287–292. doi:10.1097/NNA.0b013e3181df10d9.
14. Parul S, Ashok KB, Mercy ML. Risk assessment tool for the prediction of fall. J Nurs Health Sci. 2013;2(2):36–41.
15. Rafiey H, Soleimani F, Torkzahrani S, Salavati M, Nasiri M. Scale development and psychometrics for parents' satisfaction with developmental care in neonatal intensive care unit. Iran J Child Neurol. 2016;10(4):16–24.
16. Cooper CL, Nolt JD. Development of an evidence-based pediatric fall prevention program. J Nurs Care Qual. 2007;22(2):107–112. doi:10.1097/01.NCQ.0000263 098.83439.8c.
17. Cho MS, Song MR, Cha SK. [Risk factors for pediatric inpatient falls]. J Korean Acad Nurs. 2013;43(5):595–604.
18. Pauley BJ, Houston LS, Cheng D, Johnston DM. Clinical relevance of the Humpty Dumpty falls scale in a pediatric specialty hospital. Pediatr Nurs. 2014;40(3):137–142.
19. Kramlich DL, Dende D. Development of a pediatric fall risk and injury reduction program. Pediatr Nurs. 2016;42(2):77–82.
20. DiGerolamo K, Finn Davis K. An integrative review of pediatric fall risk assessment tools. J Pediatr Nurs. 2017;34:23–28. doi:10.1016/j.pedn.2017.02.036.
21. Kim EJ, Lim JY, Kim GM, Lee MK. Meta-analysis of the diagnostic test accuracy of pediatric inpatient fall risk assessment scales. Child Health Nurs Res. 2019;25(1):56–64. doi:10.4094/chnr.2019.25.1.56.
Keywords:

falls; pediatrics; reliability; scale; validity

© 2020 The Authors. Published by Wolters Kluwer Health, Inc.