Share this article on:

Validity of the Johns Hopkins Fall Risk Assessment Tool for Predicting Falls on Inpatient Medicine Services

Klinkenberg, W. Dean PhD; Potter, Patricia PhD, RN, FAAN

doi: 10.1097/NCQ.0000000000000210
Articles

The Johns Hopkins Fall Risk Assessment Tool (JHFRAT) is widely used but few studies have examined its psychometric properties. We examined the predictive validity of the JHFRAT for 13 574 patient admissions to medicine units at a large academic medical center in 2014. There were 204 patient falls reported. While patients who fell had higher JHFRAT total scores, a majority of patients who fell were classified by the JHFRAT as moderate or low risk.

Department of Research for Patient Care Services, Barnes-Jewish Hospital, St. Louis, Missouri (Drs Klinkenberg and Potter).

Correspondence: W. Dean Klinkenberg, PhD, Department of Research for Patient Care Services, Barnes-Jewish Hospital, 4901 Forest Park Ave, MS: 90-75-594, St. Louis, MO 63108 (wdk9159@bjc.org).

The authors are indebted to Eileen Costantinou for providing valuable insights and feedback throughout the process.

The authors declare no conflict of interest.

Supplemental digital content is available for this article. Direct URL citation appears in the printed text and is provided in the HTML and PDF versions of this article on the journal's Web site (www.jncqjournal.com).

Accepted for publication: May 22, 2016

Published ahead of print: August 1, 2016

PATIENT FALLS in hospitals are a persistent and significant health problem. Every year, between 700 000 and 1 100 000 people fall during a hospitalization.1,2 Falls that result in injuries have been found to increase time in hospitals by an average of 6.3 days3 and add thousands of dollars in additional costs that are no longer reimbursed by the Centers for Medicaid & Medicare Services.2,4 For a number of years, The Joint Commission has included reducing harm from falls for hospitalized patients as a top safety goal.5 It is recognized by The Joint Commission that since falls account for a significant portion of injuries in hospitalized patients, hospital nurses are required to assess a patient's risk for falls and take action to reduce the risk of falling as well as the risk of injury, should a fall occur.

Clearly, patient falls are a financial and regulatory burden for hospitals. For patients, falls may cause significant pain and suffering that could have been prevented. Given the burden of accurately identifying patients' fall risks and implementing prevention strategies, a number of assessment tools are being used that allow nurses to assess patients' fall risks and then categorize their overall risk level for falling6, while new fall risk assessment tools continue to be developed and tested.7,8 Once a patient's fall risk is determined, nurses can deliver interventions that target a patient's specific risk level; popular interventions for high-risk patients include increased rounding, environmental alerts, and alarms.6

Many assessment tools, however, have poor or unknown psychometric properties or are impractical for routine clinical use (eg, lengthy administration times). With these concerns in mind, a nursing team developed a new measure, the Johns Hopkins Fall Risk Assessment Tool (JHFRAT), that was based on existing evidence and brief enough for hospital-based nurses to quickly administer, score, and interpret.9 It assesses 7 risk factors that had been significantly associated with fall risk in previous studies: patient age, fall history, mobility, elimination, mental status changes, medications, and patient care equipment.

The 7 fall risk factors were developed into individual assessment items, each of which can each yield anywhere from 0 to 7 points. The individual items are summed into a total score, which is then coded into categories indicating low, medium, or high risk for a fall. For the initial version of the tool, the cutoff scores were determined by consensus opinion of the research team: low risk was defined as a total score of 5 or less; moderate risk was a total score of 6 to 10; and high risk was a total score greater than 10. The tool was pilot tested on several inpatient units, which resulted in scoring procedures being modified, so that moderate risk was defined as scores of 6 to 13 and high risk was a score of 14 or higher.

While the JHFRAT had good content validity as judged by a panel of experts,10 data on the full psychometric properties of the JHFRAT are scant. Kim and colleagues11 compared the psychometrics of the Morse Fall Scale12 (MFS), Bobath Memorial Hospital Fall Risk Assessment Scale13 (BMFRAS), and the JHFRAT.10 The study enrolled 356 inpatients from 2 units with high fall rates at 5 adult acute care teaching hospitals in South Korea; each patient's fall risk was assessed with all 3 instruments. Patients who fell (19.9% of the sample) had a significantly higher mean score than patients who did not fall on all 3 assessment tools. For the JHFRAT, patients who fell had an average score of 15.3 (±6.0) while the mean score for nonfallers was 10.6 (±6.2).

The JHFRAT had somewhat lower sensitivity (69.0%) than the MFS (78.9%) and the BMFRAS (76.1%) but slightly better specificity (60.0% vs 55.8% for the MFS and 58.3% for the BMFRAS). The 3 tests were nearly equal for positive and negative predictive values. Another indicator of a test's ability to discriminate is the value for the area under the curve (AUC) computed from an analysis plotting the receiver operating characteristic (ROC) curve. Values for AUC can range from 0.50 to 1.0, but values under 0.70 indicate poor discriminant validity.14 The JHFRAT had the lowest value for the AUC of the 3 tools in the study by Kim et al (0.71 for the JHFRAT vs 0.76 for the MFS and 0.72 for the BMFRAS).

Hnizdo and colleagues15 conducted a prospective study to test the predictive validity of a slightly modified version of the JHFRAT for 107 community-dwelling older adults, nearly a third of whom fell during the study period (30.8%). Patients who fell had a higher total score than patients who did not fall (16.3 vs 13.6). Furthermore, a higher percentage of patients who fell were initially classified as a high fall risk (a total score of 14 or higher) at study entry than patients who did not fall (63.6% vs 43.2%). The modified JHFRAT had sensitivity of 72.5% and specificity of 52.2% and an AUC of 0.66.

In spite of the limited psychometric data, the JHFRAT has been adopted by dozens of hospitals around the world.16 For this study, our goal was to address a gap in the literature by examining the predictive validity of the JHFRAT. We examined how well the Hopkins risk scores and risk categories predicted actual falls for patients on inpatient medicine floors.

Back to Top | Article Outline

METHODS

Patients

This study was conducted at a large tertiary academic medical center located in the Midwest. Following approval from the medical center's Human Research Protection office, the study team conducted a retrospective chart review, which examined data from electronic medical records (EMRs) for patients admitted in calendar year 2014 to the 8 medicine floors of the hospital.

Back to Top | Article Outline

Measures

The hospital was authorized to use the JHFRAT house-wide in spring 2012 to assess fall risk. The JHFRAT is administered for every patient at admission and again at every shift change. In the event of a fall, the JHFRAT is readministered. The JHFRAT total scores and fall risk categories are recorded in the EMR. Nurses may assign a risk category based on the JHFRAT guidelines, but they also have the discretion to use clinical judgment to assign a higher or lower risk category.

For this study, we requested the following data from the EMR for patients on the 8 medicine units: date of admission, unit admitted to, every Hopkins risk score assigned during admission along with the date and time of the assessment, and every Hopkins risk category assigned during admission along with the date and time of the assessment. We received Hopkins data for the entire inpatient episode, even if a patient was transferred from medicine to another floor.

The hospital also maintains a Safety Event Management System that includes records of patient falls. We, therefore, requested data from this system for 2014 of all patient falls that occurred on the same 8 units. Data from the EMRs and Safety Event Management System were matched on the basis of unique patient identifiers and merged into a single database. Once the data were merged, patient identifiers were removed from the database. No patient demographics were collected to maintain patient anonymity.

Back to Top | Article Outline

Data analyses

Staff nurses completed the JHFRAT for all patients at admission to the unit and again at every shift change. To ensure consistency in the time of assessments, we chose to use the first recorded JHFRAT assessment for most of the analyses in this study, which was typically that assessment completed at admission to the unit. For patients who fell, we also coded the JHFRAT risk score and risk category that immediately preceded the patient fall; all of these assessments were completed no more than 1 day prior to the fall.

We examined the predictive validity of the JHFRAT in several ways. First, we compared the initial JHFRAT total score for those who fell and those who did not fall using a t test. Second, we compared the initial JHFRAT risk category for those who fell and did not fall using a χ2 test. Third, we computed sensitivity, specificity, positive predictive value, and negative predictive value, comparing patients assessed as high risk with patients who were assessed as either moderate or low risk. Fourth, we examined the ability of JHFRAT risk categories to discriminate between those who fell and those who did not using the ROC curve. Data were analyzed using SPSS version 22 (IBM Corp., Armonk, New York).

Back to Top | Article Outline

RESULTS

We analyzed data for 13 574 patient admissions in calendar year 2014. Based on administrative data reviewed separately from the JHFRAT data, the average length of stay for patients on the 8 patient care units was 4.2 days. During their inpatient stay, patients received a mean of 8.9 (±9.8) separate JHFRAT assessments (range = 1-246), with a mean of 8.5 (±9.4) separate fall risk categories (range = 1-246) assigned.

The average JHFRAT score at admission was 6.84 (±4.99). For all patients, 10.7% were rated as a high fall risk at the initial assessment (Table); the overwhelming majority of patients were rated as moderate or low fall risk. Risk assessments, however, were not static. During the course of the entire hospitalization, 22.6% of patients were rated as a high fall risk at least once.

Nurses have discretion to assign risk categories that are independent of JHFRAT total scores, so we examined how frequently the JHFRAT score differed from the risk category at the initial assessment. All of the risk categories were within the range specified by the JHFRAT total score, although 1.2% of patients with a JHFRAT total score were not assigned a risk category.

Back to Top | Article Outline

Predictive Validity of JHFRAT

Of 13 574 patient admissions, 204 (1.5%) had a fall reported in the Safety Event Management System. Patients who fell had a significantly higher mean initial JHFRAT total score than patients who did not fall (10.2 vs 6.8; t13,562 = −9.70; P < .001). In addition, patients who fell were significantly more likely to be classified as a high fall risk at the initial assessment than patients who did not fall (27.4% vs 10.4%; χ22 = 77.71; P < .001; Table).

About 4% of patients did not have a JHFRAT assessment completed at any time prior to their fall. For patients who fell and had a JHFRAT completed before falling, 28.9% had been rated as a high fall risk on the most recent JHFRAT assessment (Table), which is only slightly higher than the 26.5% of those who fell had been assessed as high risk at admission.

Overall, the JHFRAT had low sensitivity (26.5%) but high specificity (89.6%). In addition, the JHFRAT had low positive predictive value (3.7%) but high negative predictive value (98.8%). The area under the ROC curve (AUC) was 0.69 (P < .05; 95% confidence interval = 0.66, 0.72; see Supplemental Digital Content Figure, available at: http://links.lww.com/JNCQ/A280).

Back to Top | Article Outline

DISCUSSION

Our study found that the majority of medicine patients who fell were classified as a moderate or low fall risk by the JHFRAT, whether that assessment had been completed on admission to the unit or just prior to a fall. With sensitivity of just 27.1% and specificity of 89.6%, and a value for the area under the ROC curve of 0.69, the JHFRAT is misclassifying fall risk for many patients. Taken together, these results suggest moderate to poor predictive validity for the JHFRAT. While we observed a low fall rate in this patient population (1.5% of admissions), sensitivity and specificity are values influenced by characteristics of the test itself and not by the frequency of falls. As an assessment tool predicting risk for falling, it is not clear that the JHFRAT is adding significant value in the clinical setting.

This analysis, however, assumes that the JHFRAT is consistently completed correctly. One of the JHFRAT's assessment items is a patient's mobility status, which includes 3 factors: patients' gait, whether assistance is needed with transfer and ambulation, and whether a visual or auditory impairment affects mobility. If a nurse assesses a patient in bed and simply asks a patient to self-report his or her gait and level of assistance required, the data are not objective. Instead, a more accurate assessment would require a nurse to observe a patient walk or at least transfer from a wheelchair to the bed. The mobility and level of assistance factors can account for up to 4 points in a patient's total score. If mobility status is not objectively measured, patients' JHFRAT scores, therefore, can be falsely low.

While the JHFRAT may not have desirable psychometric properties, it is possible that its routine use has created a culture of fall awareness that may have reduced the incidence of patient falls. Once the JHFRAT is completed, for example, a nurse is presented with a list of potential interventions that are to be tailored to a patient's assessed risk level. For a high fall risk patient, 1 of the options is a bed or chair alarm. For moderate risk patients, nurses may opt to place a gait belt near the patient's bed or to remain with the patient while he or she toilets. Even low fall risk patients may be offered fall prevention interventions, such as nonskid footwear or putting the patient's bed in the lowest position.

However, creating a culture of fall prevention can be achieved through many methods, not only by using a particular fall risk assessment tool such as the JHFRAT. It is also possible that, while a culture of fall prevention may have developed, JHFRAT assessments could, for some patients, underestimate actual fall risk. As we noted previously, patients who are classified as moderate or low fall risk receive less intensive fall prevention measures, and those patients were ultimately the ones who were most likely to fall.

The primary limitation of this study is that we evaluated the psychometric properties of a tool that is used in a clinical setting. There may be variation in how nurses code individual items that could affect the scale's reliability and therefore its validity; we did not have a method to evaluate the potential impact of this. Furthermore, as noted previously, the JHFRAT is used to classify fall risk and help nurses select fall prevention interventions. As such, there is some confounding in purpose between fall risk assessment and fall prevention that could affect the scale's psychometric properties.

Nevertheless, the JHFRAT has considerable weaknesses in this clinical setting. We suggest that future studies evaluate the predictive validity of other methods of assessing fall risk. For example, there is evidence that features of a patient's walk, such as gait speed, can predict risk for falling.17 Assessing gait speed may be a quicker and more reliable method for assessing fall risk in hospitalized patients and, therefore, could also have better predictive validity.

Back to Top | Article Outline

REFERENCES

1. Currie L. Fall and injury prevention. In:Hughes R, ed. Patient Safety and Quality: An Evidence-Based Handbook for Nurses. Rockville, MD: Agency for Healthcare Research and Quality; 2008:1–56.
2. Agency for Healthcare Research and Quality. Preventing Falls in Hospitals. Rockville, MD: Agency for Healthcare Research and Quality; 2013. http://http://www.ahrq.gov/professionals/systems/hospital/fallpxtoolkit/. Accessed January 19, 2016.
3. Wong CA, Recktenwald AJ, Jones ML, Waterman BM, Bollini ML, Dunagan WC. The cost of serious fall-related injuries at three Midwestern hospitals. Jt Comm J Qual Patient Saf. 2011;37(2):81–87.
4. Centers for Medicare & Medicaid Services. Center for Medicaid and State Operations. Baltimore, MD: Centers for Medicare & Medicaid Services. https://downloads.cms.gov/cmsgov/archived-downloads/SMDL/downloads/smd073108.pdf. Published July 31, 2008. Accessed January 19, 2016.
5. The Joint Commission. Hospital: 2016 National Patient Safety Goals. http://http://www.jointcommission.org/hap_2016_npsgs/. Accessed February 3, 2016.
6. Williams T, Szekendi M, Thomas S. An analysis of patient falls and fall prevention programs across academic medical centers. J Nurs Care Qual. 2014;29(1):19–29.
7. Yip WK, Mordiffi SZ, Wong HC, Ang ENK. Development and validation of a simplified falls assessment tool in an acute care setting [published online ahead of print March 18, 2016]. J Nurs Care Qual. doi:10.1097/NCQ.0000000000000183.
8. Ruroede K, Pilkington D, Guernon A. Validation study of the Marianjoy Fall Risk Assessment Tool. J Nurs Care Qual. 2016;31(2):146–152.
9. Poe SS, Cvach MM, Gartrell DG, Radzik BR, Joy TL. An evidence-based approach to fall risk assessment, prevention, and management. J Nurs Care Qual. 2005;20(2):107–116.
10. Poe SS, Cvach M, Dawson PB, Straus H, Hill EE. The Johns Hopkins Fall Risk Assessment Tool: postimplementation evaluation. J Nurs Care Qual. 2007;22(4):293–298.
11. Kim KS, Kim JA, Choi YK, et al A comparative study on the validity of fall risk assessment scales in Korean hospitals. Asian Nurs Res. 2011;5(1): 28–37.
12. Morse JM, Morse RM, Tylko SJ. Development of a scale to identify the fall-prone patients. Can J Aging. 1989;8(4):366–377.
13. Korean Hospital Nurses Association. Safety Management Guidelines for Nursing. 6th ed. Seoul, Korea: Korean Hospital Nurses Association; 2005.
14. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.
15. Hnizdo S, Archuleta RA, Taylor B, Kim SC. Validity and reliability of the modified Johns Hopkins Fall Risk Assessment Tool for elderly patients in home health care. Geriatric Nurs. 2013;34(5):423–427.
16. Institute for Johns Hopkins Nursing, Johns Hopkins University. Fall Risk Assessment. Baltimore, MD: Institute for Johns Hopkins Nursing, Johns Hopkins University. http://http://www.hopkinsmedicine.org/institute_nursing/models_tools/fall_risk.html. Accessed February 23, 2016.
17. Pamoukdjian F, Paillaud E, Zelek L, et al Measurement of gait speed in older adults to identify complications associated with frailty: a systematic review. J Geriatric Oncol. 2015; 6(6):484–496.

fall risk assessment; inpatients; Johns Hopkins Fall Risk Assessment Tool; predictive validity; psychometric studies

Supplemental Digital Content

Back to Top | Article Outline
Copyright © 2017 Wolters Kluwer Health, Inc. All rights reserved