ACTIVELY involving patients in their care can improve patient safety. To accomplish this, nurses seek efficient and effective ways to engage and educate patients. Previous efforts have found patient engagement associated with higher patient satisfaction and a reduction in medical errors.1 , 2
The Veterans Health Administration (VHA), the largest national integrated health care system in the United States,3 has developed a novel approach to patient engagement to promote safety: The Daily Plan (TDP). In 2007, the VHA began piloting TDP, a personalized document outlining each patient's scheduled interventions for the day, which a nurse reviews at the bedside with the patient and family members. The document includes printed information such as an up-to-date medication schedule, allergies, scheduled tests and procedures, upcoming health care appointments, and discharge planning information.4
An initial pilot evaluation at 10 VHA pilot sites showed that TDP increased recognition in discrepancies in the plan of care by nurses, patients, and family members.5 Specifically, 17.6% of nurses and 47.5% of patients and families found a discrepancy between the physician's report and TDP. On the basis of this initial success, TDP was offered to Veterans Affairs (VA) medical centers nationwide beginning in 2008 and sites across the nation began to implement TDP between 2008 and 2015.
Although TDP demonstrated value in preventing inpatient errors and has spread to VHA facilities nationally, the effects on hospital outcomes such as length of stay (LOS) and readmissions are unknown. Therefore, we developed an observational cohort study to assess whether implementation of TDP was associated with improvements in these measures. This study is unique in that it offers a rigorous analysis of a quality improvement/patient safety effort by not only examining pre-/postoutcomes for the intervention group but by also analyzing outcomes for matched controls during the same time frame. Such a method provides a high standard for evaluating quality improvement interventions.
This was a retrospective cohort study using a matched control group from the same period comparing changes in LOS and 30-day readmission rates between medical surgical units that did and did not implement TDP between calendar years 2008-2015.
Between December 1, 2008, and September 28, 2015, 83 inpatient medical and surgical units from 19 VA medical centers across the United States implemented TDP for at least 3 months. Forty-seven units employed TDP for the entire study period, and 36 units ceased TDP use before the study end date. Units that prematurely discontinued TDP mainly did so because of logistical difficulty in implementation (from either a technological standpoint or a staffing shortage) or complaints of redundancy, confusion, or disinterest from patients. Even the 36 units that discontinued TDP before the study end date were included in the analysis because some effects of the intervention may have lingered.
Units were primarily located in high-complexity facilities (Supplemental Digital Content, Table 1 available at: http://links.lww.com/JNCQ/A350). Medical and surgical units identified in TDP implementation records were matched to electronic medical record (EMR) data. The patients admitted to these TDP-implementing units formed the study cohort; patient stays were used as the unit of analysis.
Control cohort development: Comparison units and comparison stays
The selection of controls for each patient stay involved a 2-step process. First, the study team identified control units comparable to those that used TDP. For each of the 19 VA medical centers in the study, analysts selected up to 4 control medical centers that met our criteria. When possible, analyses excluded medical centers that had any TDP units to minimize cross-contamination because of shared staffing. This resulted in 31 medical centers from which analysts identified 2110 medical and surgical units that never implemented TDP. Analysts narrowed the list to 319 non-TDP units by selecting up to 4 matched non-TDP units per TDP unit using random sampling.
Second, analysts identified patient stays in the control units comparable to those in TDP units. To assess the impact of TDP, the study team constructed a propensity score (PS) using a selected population after the matched units were selected.6 , 7 Patients on TDP units may have different characteristics than those on control units. The PS was estimated using a logistic regression model and defined as the probability of being on a TDP unit based on patient characteristics and medical history. Variables included in the logistic regression are confounders that can simultaneously explain the probability of being on a TDP unit and the medical outcomes (ie, LOS, readmission, and 30-day mortality).
In addition to matching on units, analysts applied inverse probability of treatment weight (IPTW) method8 , 9 to adjust for potential confounding because of unmeasured factors associated with TDP, for example, leadership or culture on the units. This is a weight-based method to account for unknown differences. As a result, patients who were admitted onto TDP units but had a low predicted probability for such admission (based on the PS) would receive a higher weight, whereas patients who had a high predicted probability being admitted to a TDP unit received a lower weight. The same rationale was applied to patients admitted to the control units. IPTW was then used to weigh observations when comparing the different study outcomes between the TDP and control units. Residual confounding—despite the matching protocol—could be reduced by this approach.
Analyses identified 16 833 patient stays across all TDP and control units. For each patient stay, the study team collected information on LOS, readmission within 30 days of discharge (if any), primary discharge diagnosis, secondary discharge diagnoses (if any), and patient demographics (age, race/ethnicity, sex; Supplemental Digital Content, Tables 2 and 3 available at: http://links.lww.com/JNCQ/A351 and available at http://links.lww.com/JNCQ/A352).
Descriptive and inferential statistics were used to compare outcomes (LOS, 30-day, and readmission rate) for each unit before and after TDP implementation. A 3-month implementation phase was excluded from the analysis to account for staff education and training. The preimplementation phase was defined as 12 months before TDP start date, and the postimplementation phase was defined as 36 months after the implementation phase. Analysis included linear regression of LOS on phase (preimplementation vs postimplementation), clustering by medical center to account for shared staffing.
A Poisson model was used to compare 30-day readmission rates for each unit before and after TDP implementation. Three primary discharge diagnoses defined study outcomes (acute myocardial infarction, congestive heart failure, and pneumonia); analyses combined all 3 of these.
Analyses included 119 724 patient stays from 83 TDP units (Supplemental Digital Content, Table 1 available at: http://links.lww.com/JNCQ/A350) and 319 control units, with a roughly 4:1 control-to-intervention ratio. Although TDP units and control units were matched, there were still substantial differences between the patients on these units regarding African American, Hispanic, and white race, mean age and the percentage of younger patients, and hemoglobin A1C concentration of greater than 9 (Supplemental Digital Content, Table 2 available at: http://links.lww.com/JNCQ/A351). After adjustment using IPTW, these differences in hospital stays were greatly reduced, with substantial variation remaining only in mean hemoglobin A1C concentration.
After combining all 3 outcome categories (acute myocardial infarction, congestive heart failure, and pneumonia), there were a total of 1474 patient stays in the control units and 3148 patient stays in the TDP units during the preimplementation phase. There were 3787 patient stays in the control units and 8424 patient stays in the TDP units in the period postimplementation.
For the TDP units, the mean LOS went from 5.6 days during the implementation phase to 5.0 days during the postimplementation phase, for an average decrease of 0.6 day per patient stay (P = .025). During the same time, after matching on unit characteristics and using IPTW to compare patient stays with similar severity and complexity, the control units had a decrease of 0.7 day per patient stay (P = .05), from 5.5 to 4.8 days, from pre to postimplementation. There was no significant net advantage on the TDP units (P = .99), as they had a decrease of 0.6 day per patient stay (Figure). In other words, LOS decreased among both TDP and control units, and there was no statistically significant difference between their margins of reduction. In a multivariate analysis that compared TDP units with their control units, we found that each additional 3 months of TDP use was associated with a 0.1-day (P < .001) drop in LOS.
The readmission rate increased for both control and TDP units. The preimplementation rates were 21.3% in the control units and 21.7% in the TDP units, and postimplementation rates were 25.0% and 22.5%, respectively. There was an absolute, instead of relative, increase of 3.7% (P = .02) in the control units and a 0.9% (P = .37) increase in the TDP units. Thus, there was no observed impact of TDP on readmission rates.
We found that TDP was implemented widely across the VHA, impacting over 16 000 medical and surgical admissions over 8 years. The original intent of TDP was to improve patient safety by reducing adverse events because of lacking communication. Some examples are medication errors, incorrect diagnostic testing being done or performed on the wrong patient, and delays or omissions of treatments such as physical therapy. Even though the primary aim of TDP was to improve safety, we also wanted to assess for unintended benefits or consequences.
We did not find any advantage or disadvantage associated with TDP in regard to LOS. This is perhaps due to overall trends toward decreased LOS on all medical and surgical units, including those in our control group. It is possible that there is limited room to gain further efficiencies in LOS. Although LOS decreased on both intervention and control units, 30-day readmission rates remained largely unchanged across units.
Although efficiency (as measured by LOS and readmission) was not improved, it also did not worsen. Because previous work has shown the patient safety benefits of TDP,5 the nursing implications are that we can confidently implement TDP to improve safety without concern about unintended consequences to efficiency such as LOS and readmission. There may have been unexpected benefits from TDP that were not measured in this study. It is possible, for example, that nurses on TDP units were more satisfied than those on control units because of increased patient connection. It is also possible that other health care team members, such as physical therapists or physicians, would report improved experiences as well, because TDP allows the plan of care to be implemented with more transparency.
The ideal study would include a detailed, qualitative review of mistakes that were avoided because of TDP. Given that TDP has already been widely implemented within the VHA, any study evaluating the practice would be the strongest if involving matched controls. Incorporating observational data collection, though a more time- and resource-intensive method than a purely retrospective data analysis, might provide invaluable insight to future researchers evaluating TDP. Continued high-quality evaluations of quality improvement and patient safety efforts are crucial to ensure we accurately measure our ultimate outcome of interest: patient safety.
Nevertheless, to our knowledge, this is the first large study to use a matched control group from the same period, adjusted with IPTW to evaluate a quality improvement intervention. We believe that our approach represents an improvement over mirror image (pre-/post-) studies more typically employed in the quasiexperimental evaluation of quality improvement interventions.
This study has several limitations. It is important to acknowledge that LOS and 30-day readmissions rates are influenced by multiple factors. Although we did adjust for patient, unit, and medical center factors, it is possible that unmeasured confounders such as differences in nursing skill and experiences or patients' educational level and family resources led to type 1 error (false positive) in evaluating reduction in LOS, or type 2 (false negative) error in evaluating reductions in 30-day readmission. Although our matching and IPTW approach is designed to approximate the advantages of randomization in controlling for confounding, only a randomized trial of TDP would fully control for unmeasured differences.
Once a unit adopted TDP, the staff was free to implement it in whichever manner most appropriate and effective. Some units identified optimal fonts and formats for their patients. Some incorporated only specific sections of TDP, or used it to target only specific categories of patients. It is difficult to determine whether standardization could increase the impact of the intervention, but it appears that tailored adoption of TDP led to longer implementation. It must be noted that the applicability of this study to the general population may be limited; patients who receive treatment from the VHA have been found to differ significantly from patients in the private sector.
TDP has previously been shown to improve communication between nurses and patients about inpatient care, to improve patient safety, and to generate patient satisfaction. Although this study did not indicate a decrease in LOS or readmission for units that used TDP versus matched controls, TDP warrants additional large-scale qualitative analysis to specifically target patient safety-related outcomes.