Secondary Logo

Journal Logo

Validation of the BOOST Risk Stratification Tool as a Predictor of Unplanned 30-Day Readmission in Elderly Patients

Sieck, Carol, PhD, RN; Adams, William, PhD; Burkhart, Lisa, PhD, RN, ANEF

Quality Management in Healthcare: April/June 2019 - Volume 28 - Issue 2 - p 96–102
doi: 10.1097/QMH.0000000000000206
Methods and Instruments
Free
SDC

Introduction: Risk stratification tools can identify patients at risk for 30-day readmission, but available tools lack predictive strength. One of these tools is the Better Outcomes by Optimizing Safe Transitions (BOOST) 8 P's tool.

Objectives: The primary objective of this study was to validate the 8 P's tool as well as measure the predictive strength of variables within this tool.

Methods: This was a quantitative study that included 1 year of hospitalized elderly patients (n = 6849). Odds ratios were used to determine the strength of the association between variables individually with readmission. Multivariable logistic regression was used to evaluate the predictive strength of the BOOST risk stratification tool.

Results: This study demonstrated that 5 of the 8 variables in the BOOST risk stratification tool showed significant association with 30-day readmission including the variables of health literacy (P = .030), depression (P = .003), problem medications (P = .001), physical limitations (P ≤ .001), and prior hospitalization (P ≤ .001). Combining variables using multivariable logistic regression, the BOOST 8 P's tool had limited predictive capability with a C-statistic of 0.631.

Conclusion: This study was the first attempt to validate the BOOST 8 P's tool and to utilize nursing documentation within an electronic medical record to capture social determinants of health.

American Society of Plastic Surgeons, Arlington Heights, Illinois (Dr Sieck); and Loyola University Health System, Loyola University of Chicago, Maywood, Illinois (Drs Adams and Burkhart); Center of Innovation for Complex Chronic Healthcare, Hines VA Hospital, Hines, Illinois (Dr Burkhart).

Correspondence: Carol Sieck, PhD, RN, American Society of Plastic Surgeons, 444 E Algonquin Rd, Arlington Heights, IL 60005 (tcsieck@aol.com).

The authors declare no conflicts of interest.

The US population shift toward a greater number of seniors who have chronic illness and require more care has challenged health care systems to provide high-quality care while minimizing the risk of adverse events.1,2 The lack of coordination posthospitalization to home has led to preventable and costly unplanned hospital readmissions.3 Medicare has estimated the burden of 30-day readmission hospital penalties to be more than 41 billion annually, impacting 3.3 million adults, with the majority, or 55.9%, being Medicare patients.3 One solution to this problem is to identify chronically ill patients during vulnerable periods when they have complex or intensive needs (eg, posthospitalization) using risk stratification tools.5 Clinicians have been unable to accurately predict patients at risk for readmissions without predictive tools.6

Several tools have been reported to predict 30-day readmissions. As shown in Table 1, each tool includes different predictive variables, with most measuring diagnosis, demographic, and health utilization. Few include social determinants or contextual factors related to health, which have been shown to be associated with 30-day readmission. These include poverty, lack of medical insurance, limited education, poor health literacy, social isolation, substance abuse, mental illness, and discharge to a community that has poor access to health care (eg, rural setting).5,17

Table 1

Table 1

As shown in Table 1, 4 risk stratification tools were designed to predict readmission but demonstrated poor to moderate predictive validity (0.78-0.66 C-statistics). Risk stratification tools are typically evaluated for predictive strength using C-statistics, which quantifies the proportion of time the model correctly identifies high and low risks: 0.50 indicating that the model is equivalent to chance; 0.70 to 0.80 indicating modest discriminative ability; and a threshold of 0.80 demonstrates good discriminative ability.5,18 Another 6 tools have been used to predict 30-day readmission but were designed to predict something else. The collective predictive strength of these risk tools for readmission was modest to strong, yet none emerged as the gold standard for predicting readmission.

Variables associated with unplanned readmissions are complex and include a range of medical, social, and economic factors.5,19 Social determinants of health improve predictive strength of risk assessment tools, but few are used because of the difficulty identifying social factors easily within the electronic health record (EHR).7,20

One risk stratification tool that integrates social determinants is the Better Outcomes by Optimizing Safe Transitions (BOOST) 8 P's instrument, which combines 8 social, functional, psychological, and medical factors. This tool provides a more holistic risk assessment.5,21 There was only one published study that measured the use of the BOOST risk stratification tool, but there was no information on the predictive strength of the tool for 30-day readmissions.22 Therefore, the aim of this study was to determine the predictive validity of the BOOST risk stratification tool for elderly (65+ years) hospitalized patients.

Back to Top | Article Outline

METHODS

This was a descriptive, retrospective, quantitative study using secondary data from the EHR to measure the degree to which each of the 8 P variables in the BOOST predicted 30-day unplanned hospital readmission. The sample included hospitalized elderly patients who were admitted to an academic Midwest hospital over 1 year from January 1, 2016 to December 31, 2016. Inclusion criteria were patients older than 65 years who were hospitalized during the study period. Exclusion criteria were patients who were admitted for an elective hospitalization, observation care, emergency care only, inpatient admission for psychiatry, as well as if the patient expired, left against medical advice, had a permanent address outside the state, or was transferred to another hospital or long-term care facility.23 Full institutional review board approval was obtained from the academic hospital used in the study.

The dependent variable was unplanned hospital readmission within 30 days of discharge from an index hospital admission from January 1, 2016 to December 31, 2016. An initial 3-month data search indicated that 1 year would ensure an adequate sample of readmitted and nonreadmitted seniors.

The independent variables are the 8 P's included in the BOOST risk stratification tool.21 These are nominal binary variables (met/unmet) during the index admission.21,24 The health system did not collect BOOST items explicitly, but proxy measures for each BOOST item were available in the EHR. Three informatics documentation experts were consulted to identify the proxy measures, and each variable was searched to ensure data were documented. BOOST items are described as follows:

  • Problem medication is conceptually defined as poly-pharmacy OR high-risk medications.21 This was operationalized in the discharge order set as either the use of 10 or more routine medications or 1 HEDIS-identified high-risk medication (http://www.ncqa.org/hedis-quality-measurement/hedis-measures/hedis-2016/hedis-2016-ndc-license/hedis-2016-final-ndc-lists).
  • Psychological problems is defined as a diagnosis or history of depression.21 This was operationalized as any documentation in the patient record of a diagnosis or history of depression using International Classification of Diseases, Tenth Revision (ICD-10) codes during the index admission.
  • Principal diagnosis is defined as the patient having a diagnosis of cancer, stroke, diabetes, chronic obstructive pulmonary disease (COPD), and/or heart failure.21 This was operationalized as any documentation in the hospital discharge billing records for cancer, stroke, diabetes, COPD, and/or heart failure using ICD-10 codes for the index admission. Problem lists were not used, as they are not regularly updated and therefore inaccurate.
  • Physical limitation is defined as deficits in activities of daily living, medication administration, and organizing follow-up with their primary physician.21 This was operationalized as (1) presence of referrals for physical, occupational, or nutritional therapy; or (2) nurses notes indicating “mobility issues” on the “Functional Assessment” form.
  • Poor health literacy is defined as inability of the patient, family, or caregivers to perform a “teach back” of discharge education.21 This was operationalized as (1) use of teach back as an education method on the Patient Education form with an outcome of “no evidence of learning” or “needing reinforcement”; (2) documentation indicating that there were barriers to learning and/or the need for an interpreter on the nursing Learning Assessment Sheet; or (3) documentation of compromised learning readiness (nonacceptance” or “refusal”) in the nurse's notes.
  • Lack of Patient Support is defined as the absence of a reliable caregiver to assist with the discharge, living alone or without needed assistance, and/or lack of connection to the primary care provider.21 This was operationalized as (1) being single or widowed, (2) documentation indicating “patient lived alone,” “no one would help the patient at home after hospitalization,” or “the patient was not receiving care from health care agencies” on the Domicile Problems Assessment Sheet.
  • Prior hospitalization is defined as documentation of a previous hospitalization prior to the index readmission within the past 6 months.21 This is operationalized as a previous nonelective hospital admission within 6 months of the index admission.
  • Palliative care is defined as an anticipated death within an upcoming year or an advanced/progressive illness.21 This was operationalized as physician orders or notes related to palliative care or hospice services.

Univariable logistic regression models were used to estimate the odds of 30-day readmission as a function of patients' age, sex, race, ethnicity, and individual BOOST statues. For the effect of increasing age on the odds of readmission, the linearity assumption between age and the log odds of readmission was assessed using a Hosmer and Lemeshow24 goodness-of-fit test. Subsequently, we fit a multivariable logistic regression model to estimate the adjusted odds of readmission as a function of patients' BOOST statuses while controlling for their age, sex, race, and ethnicity.

To calculate and compare the areas under the curves (C-statistics), we plotted 2 receiver operating characteristic curves: one comprising all covariates and another comprising only patients' demographics, including age, sex, race, and ethnicity. Using the DeLong et al25 method, we compared C-statistics for the full model that includes both patient demographics as well as their BOOST statuses against a reduced model that includes only patients' age, sex, race, and ethnicity information. All analyses were completed using SAS, version 9.4 (SAS Institutes, Cary, North Carolina).

Back to Top | Article Outline

RESULTS

The sample included 6894 patients 65 years or older who were hospitalized at least once during the 2016 calendar year, as shown in Table 2. About half of the patients were female (51.1% overall, 51.3% not readmitted, 50.1% readmitted) and most identified as non-Hispanic white (74.9% overall, 75.2% not readmitted, 73.1% readmitted). The average age overall was 75.28 years (SD = 7.95), for not readmitted 75.33 years (SD = 7.98), and for readmitted 75.03 years (SD = 7.84). Sixteen percent of the patients (n = 1083) were readmitted within 30 days of an index admission. Patient characteristics were not statistically significant predictors of 30-day readmission (all P values >.05).

Table 2

Table 2

Univariable analysis indicated that 5 of the BOOST variables were positively associated with readmission. Relative to those without, Patients with Problem medications had 1.57 (95% CI, 1.37-1.80) times higher odds of being readmitted; similarly, those with Psychological problems (ie, depression) had 1.61 (95% CI, 1.37-1.90) times higher odds of being readmitted (both P values <.001). Furthermore, patients with Physical limitations had 1.62 (95% CI, 1.42-1.85) higher odds of being readmitted, and those with Poor health literacy had 1.22 (95% CI, 1.03-1.44) times higher odds of being readmitted (both P values <.001). Patients with a Prior hospitalization had 2.08 (95% CI, 1.82-2.37) times higher odds of being readmitted. Controlling for age, sex, race, and ethnicity, the same BOOST indicators remained significant predictors of 30-day readmission (Table 3).

Table 3

Table 3

The area under the curve (C-statistic) was higher for the model comprising patients' BOOST statuses as well as their demographics (ie, age, sex, race, and ethnicity). For this complete model, the C-statistic was 0.63 (95% CI, 0.61-0.65), which was approximately 0.10 (95% CI, 0.08-0.13) units higher than the reduced model comprising only patient demographics (P < .001) (Figure 1).

Figure 1

Figure 1

Back to Top | Article Outline

DISCUSSION

This study explored the predictive validity of the BOOST risk stratification tool using proxy measures to operationalize each of the BOOST variables. This study was unique in that it capitalized on current established documentation workflow for data capture. Although the EHR did not directly measure the BOOST items, existing documentation practices operationalized all the variables. This minimized data collection error in introducing a new BOOST documentation form for research purposes.

Although the BOOST demonstrated poor predictive validity, findings revealed some interesting patterns. Five BOOST items were consistent with the literature in the ability to predict readmission: Problem medications,26,27Psychological depression,28Physical limitations,29,30Poor health literacy,31 and Prior hospitalization.32,33 However, 3 variables were inconsistent with the literature.

Findings indicated that Principal diagnosis lacked predictive validity, although 9 current risk stratification tools included this variable and research supports this relationship.34–36 This deviation may be due to the definition of the BOOST item as the “principal diagnosis.” Since data were extracted using billing information at any time during the index hospitalization, patients may have been admitted for secondary issues or complications exacerbated by the underlying pathology of their chronic illness or the immunologic effects of an advanced chronic illness not captured in primary diagnosis billing codes. This suggests that the conceptual definition of the BOOST item, Principal diagnosis, requires clarification.

Although the literature indicates that a lack of Patient support is associated with readmission, findings demonstrated the reverse relationship. That is, being single, widowed, living alone, and lack of help or home health services postdischarge were associated with fewer readmissions. This suggests a need for more family discharge teaching and follow-up. This may also indicate that patients without social supports may avoid health care services during the 30-day period; longitudinal research is needed to support this possibility.

This study failed to support the ability of palliative or hospice care to predict 30-day readmission. This is inconsistent with research demonstrating an association between palliative care and hospital readmissions for patients nearing the end of life.21,37,38 This lack of significance may be due to the low number of palliative/hospice referrals: 100 of 6894 patients received palliative or hospice care. Only patients who agreed to receive palliative or hospice care were included; it is unclear how many patients were eligible or could benefit from palliative or hospice care. It is difficult to evaluate the ability of this BOOST item to predict readmission, but few patients receive palliative and hospice care. Research has demonstrated that palliative/hospice care programs are associated with more appropriate use of health care resources when outpatient palliative care38 or inpatient consults on hospitalization for patients and their families facing end-of-life issues are offered.37 These findings support the need for additional research in palliative/hospice care referral decision-making processes and postdischarge care to avoid 30-day readmissions, particularly as it relates to outpatient pain management and end-of-life hospitalizations.

Back to Top | Article Outline

CONCLUSION

This study supported both medical and social variables in 30-day readmission predictive models, including problem medications, psychological (depression), physical limitations, poor health literacy, and prior hospitalization. More research is needed to better understand the complexity of social support posthospitalization. This complexity also suggests that more transitional care, specifically care coordination for vulnerable patients, could reduce 30-day readmissions. The BOOST tool kit, which includes interventions associated with each of the 8 P's, could provide transitional care guidance. More research is needed to evaluate those interventions, particularly as they relate to social supports. Integrating the BOOST risk stratification variables into the EHR could facilitate this research. Given the growing burden of seniors with chronic illness and the national move to value-based care, application of risk stratification tools in EHRs and subsequent research should guide health care practices to reduce unplanned 30-day readmissions.

Back to Top | Article Outline

REFERENCES

1. Centers for Disease Control and Prevention. The State of Aging & Health in America. Atlanta, GA: Centers for Disease Control and Prevention; 2013.
2. Naylor MD, Hirschman KB, O'Connor M, Barg R, Pauly MV. Engaging older adults in their transitional care: what more needs to be done? J Comp Eff Res. 2013;2(5):457–468. doi:10.2217/cer.13.58.
3. Horwitz L. Hospital-wide All-Cause Risk Standardized Readmission Measure: Measure Methodology Report. New Haven, CT: Yale New Haven Health Services Corporation/Center for Outcomes Research & Evaluation; 2011.
4. Hines A, Barrett M, Jiang J, Steiner C. Conditions With the Largest Number of Adult Hospital Admissions by Payer. Rockville, MD: Agency for Healthcare Research and Quality; 2014. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb172-Conditions-Readmissions-Payer.jsp. Accessed November 1, 2017.
    5. Kansagara D, Englander H, Salanitro A, et al Risk prediction models for hospital readmission: a systematic review. JAMA. 2013;306(15):1688–1698.
    6. Allaudeen N, Schnipper J, Orav E, Wachter R, Vidyarthi A. Inability of providers to predict unplanned readmissions. J Gen Intern Med. 2011;26(7):771–776.
    7. Choudhry S, Li J, Davis D, Erdmann C, Sikka R, Sutariya B. A public-private partnership develops and externally validates a 30-day hospital readmission risk prediction model. Online J Public Health Inform. 2013;5(2):219. http://ncbi.nlm.nih.gov/pubmed/24224068. Accessed November 1, 2017.
    8. van Walraven C. LACE+ Index: extension of a validated index to predict early death or urgent readmission after hospital discharge using administrative data. Open Med. 2012;6(3):e80–e90.
      9. van Walraven C, Dhalla IA, Bell C, et al Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. CMAJ. 2010;182(6):551–557. doi:.
      10. Amarasingham R, Moore B, Tabak Y, et al An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48(11):981–988.
      11. Charlson M, Pompei P, Ales K, MacKenzie C. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383.
      12. Weiner J, Starfield B, Steinberg E, Steinwachs D. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care. 1991;29(5):452–472.
      13. Naessens JM, Strobel RJ, Finnie DM. Effect of multiple chronic conditions among working-age adults. Am J Managed Care. 2011;17(2):118–122.
      14. Minnesota Department of Human Services, M. Minnesota Health Care Programs (MHCP). http://www.dhs.state.mn.us/main/idcplg?IdcService=GET_DYNAMIC_CONVERSION&RevisionSelectionMethod=LatestReleased&dDocName=dhs16_151292. Published 2011. Accessed November 1, 2017.
        15. Boult C, Dowd B, McCaffrey D, Loult L, Hernandez R, Keulewitch H. Screening elders for risk of hospital admission. J Am Geriatr Soc. 1993;41:811–817.
        16. Mosley DG, Peterson E, Martin DC. Do hierarchical condition category model scores predict hospitalization risk in newly enrolled Medicare Advantage participants as well as probability of repeated admission scores? J Am Geriatr Soc. 2009;57:2306–2310.
        17. Hu J, Gonsahn M, Nerenz D. Socioeconomic status and readmissions: evidence from an urban teaching hospital. Health Aff (Millwood). 2014;33(5):778–785.
        18. Steyerberg E, Vickers A, Cook N, et al Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–138.
        19. Haas LR, Takashi PY, Shah ND, et al Risk-stratification methods for identifying patients for care coordination. Am J Managed Care. 2013;19(9):725–732.
        20. Pantell M, Rehkopf D, Jutte D, Syme L, Balmes J, Adler N. Social isolation: a predictor of mortality comparable to traditional clinical risk factors. Am J Public Health. 2013;103(11):2056–2062.
        21. Society of Hospital Medicine. Project BOOST implementation toolkit. Retrieved from http://www.hospitalmedicine.org. Published 2015. Accessed November 1, 2017.
        22. Williams MV, Li J, Hansen LO, et al Project BOOST implementation: lessons learned. South Med J. 2014;107(7):455–465.
        23. Berkman ET, Reise SP. A Conceptual Guide to Statistics Using SPSS. Los Angeles, CA: SAGE; 2012.
        24. Hosmer D, Lemeshow S. Applied Logistic Regression. 2nd ed. New York, NY: John Wiley & Sons; 2000.
        25. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845.
        26. Marcum ZA, Handler SM, Boyce R, Gellad W, Hanlon JT. Medication misadventures in the elderly: a year in review. Am J Geriatr Pharmacother. 2010;8(1):77–83. doi:10.1016/j.amjopharm.2010.02.002.
        27. Santos AP, Silva DT, Alves-Conceicao V, Antoniolli AR, Lyra DP Jr. Conceptualizing and measuring potentially inappropriate drug therapy. J Clin Pharmocol Ther. 2005;40(2):167–176. doi:10.1111/jcpt.12246.
        28. Cancino RS, Culpepper L, Sadikova E, Martin J, Jack BW, Mitchell SE. Dose-response relationship between depressive symptoms and hospital readmission. J Hosp Med. 2014;9(6):358–364. doi:10.1002/jhm.2180.
        29. Agarwal E, Ferguson M, Banks M, et al Malnutrition and poor food intake are associated with prolonged hospital stay, frequent readmissions, and greater in-hospital mortality: results from the Nutrition Care Day Survey 2010. Clin Nutr. 2010;32(5):737–745. doi:10.1016/j.clnu.2012.11.021.
        30. Craven E, Conroy S. Hospital readmissions in frail older people. Rev Clin Gerontol. 2015;25(2):107–116. doi:10.1017/S0959259815000064.
        31. Cloonan P, Wood J, Riley JB. Reducing 30-day readmissions. J Nurs Adm. 2013;43(7/8):382–387. doi:10.1097/NNA.0b013e31829d6082.
        32. Garrison GM, Mansukhani MP, Bohn B. Predictors of thirty-day readmission among hospitalized family medicine patients. J Am Board Fam Med. 2013;26(1):71–77. doi:10.3122/jabfm.2013.01.120107.
        33. Hummel SL, Katrapati P, Gillespie BW, Defranco AC, Koelling TM. Impact of prior admissions on 30-day readmissions in Medicare heart failure inpatients. Mayo Clin Proc. 2014;89(5):623–630. doi:10.1016/j.mayocp.2013.12.018.
        34. Ford ES. Hospital discharges, readmissions, and ED visits for COPD or bronchiectasis among US adults: findings From the Nationwide Inpatient Sample 2001-2012 and Nationwide Emergency Department Sample 2006-2011. Chest. 2015;147(4):989–998. doi:10.1378/chest.14-2146.
        35. Hijjawi SB, Abu Minshar M, Sharma G. Chronic obstructive pulmonary disease exacerbation: a single-center perspective on hospital readmissions. Postgraduate Med. 2015;127(4):343–348. doi:10.1080/00325481.2015.1015394.
        36. Linden A, Butterworth SW. A comprehensive hospital-based intervention to reduce readmissions for chronically ill patients: a randomized controlled trial. Am J Manag Care. 2014;20(10):783–792.
        37. Nelson C, Chand P, Sortais J, Oloimooia J, Rembert G. Inpatient palliative care consults and the probability of hospital readmission. Permanente J. 2011;15(2):48–51.
        38. Ranganathan A, Dougherty M, Waite D, Casarett D. Can palliative home care reduce 30-day readmissions? Results of a propensity score matched cohort study. J Palliat Med. 2013;16(10):1290–1293. doi:10.1089/jpm.2013.0213.
        Keywords:

        chronic illness; risk stratification; seniors; social determinants of health; 30-day hospital readmission; validation

        © 2019Wolters Kluwer Health | Lippincott Williams & Wilkins