Unscheduled return visits to the emergency department (ED) within 72 hours of discharge represent a substantial burden for children, families, and ED providers. Unscheduled 72-hour ED returns account for 4% of all ED visits.1–4 On a national scale, this return rate corresponds to greater than 1,000,000 additional pediatric ED visits each year.5 However, only 19%–30% of patients returning within 72 hours of their initial ED visit require hospital admission; these data suggest that a substantial proportion of return visits are preventable or potentially unnecessary.1–3,6 Additionally, children returning for ED care frequently incur unnecessary testing, treatment, and even avoidable hospitalization, resulting in increased patient costs, length of stay, and ED overcrowding.7 Thus, 72-hour return visits represent an important quality indicator and benchmark for ED care.
Prior studies have demonstrated that poor communication at discharge and inadequate understanding of discharge instructions by patients and caregivers contribute to unscheduled return visits.8,9 The pediatric ED discharge process represents a high-risk transition of care as it requires the transfer of critical information to caregivers with variable levels of health literacy. Subsequently, ED physicians rely on caregivers to comply with recommended treatments and establish follow-up care as instructed at discharge. However, after ED evaluation and discharge, many caregivers are unable to report their diagnosis, home care instructions, or follow-up recommendations. Therefore, they fail to follow recommendations as given at discharge.9–11 Prior studies associate simplification of written discharge instructions, and standardization of written and verbal instructions with increased comprehension by families and caregivers.12,13 In a prior study, caregivers of patients discharged after inpatient hospitalization reported a need for comprehensible verbal and written information, time for questions, and clear home management and follow-up plans.14 Additional studies suggest similar interventions for improving discharge processes for children evaluated in the pediatric ED.14–16
This quality improvement (QI) project aimed to decrease the unscheduled 72-hour return visit rate to a network of pediatric EDs and urgent cares (UCs). The study team created a multidisciplinary team to evaluate current processes, identify key drivers of ED/UC discharge, and implement effective interventions aimed at discharge process improvement using QI methodology.
A multidisciplinary team conducted this QI initiative at a tertiary care, university-affiliated free-standing pediatric health system composed of 2 academic EDs and 4 UCs within a large metropolitan area. All sites within the health system share the same electronic health record (EHR; Epic Systems, Verona, WI), protocols for care, and staff. Annual patient volume across all ED/UC sites is approximately 124,300. The QI team defined an unscheduled 72-hour return as a visit for a child who was initially evaluated and discharged from any affiliated ED/UC site and returned within 72 hours to the same or different site. The QI team excluded patients who were initially evaluated at an outside institution or presented for an expected return visit (eg, oxygen check or suture removal). Baseline return visits accounted for approximately 3.5% of all ED/UC visits regardless of the season. The baseline admission rate for all returning patients was 27%. The team used Standards for Quality Improvement Reporting Excellence guidelines to report the nature and effectiveness of interventions. The study was approved by the institution’s Organizational Research Risk and Quality Improvement Review Panel under agreement from the Colorado multi-institutional review board.
A multidisciplinary team of pediatric emergency medicine specialists, pediatricians, advanced practice providers, nurse managers, nurses, nurse educators, information technology experts, and a case manager and health literacy expert from all affiliated sites convened to analyze the current ED/UC discharge process and identify areas for improvement. This team performed a cause-and-effect analysis and created a Fishbone Diagram to identify aspects of the ED/UC discharge process that were potentially contributing to unscheduled 72-hour return visits (Fig. 1). An institutional survey of patient caregivers presenting for 72-hour return visits throughout the ED/UCs network was combined with existing literature and multidisciplinary content expertise to identify key drivers of 72-hour return visits. The team then used these key drivers to develop discharge focused interventions (Fig. 2).9 These interventions included (1) standardization of EHR discharge workflow, (2) creation of health literate and abbreviated discharge instructions, and (3) implementation of the teach-back method for delivery of patient education.17 The team identified, implemented, and evaluated interventions using sequential Plan-Do-Study-Act (PDSA) cycles. The multidisciplinary team met regularly to ensure effective implementation of interventions, provide and assess feedback, and address unanticipated challenges. Site-specific champions engaged staff at each ED/UC site to encourage the use of interventions and address staff apprehension to change. The team provided regular updates and monthly 72-hour return rates to ED/UC staff at clinician staff meetings, nursing huddles, and electronically via the ED/UC network email distribution list.
EHR Discharge Standardization
The first intervention involved the redesign of the EHR discharge workflow using a discharge navigator and the creation of discharge smart sets. The previous discharge model required multiple contact points between the provider and nurse and required providers to complete orders within separate EHR windows for discharge orders, instructions, medications, and follow-up. The new discharge navigator included a single screen discharge functionality to guide providers through an efficient workflow and smart sets, which are order sets that included disease-specific medications and instructions for follow-up and discharge. This improvement to the discharge workflow streamlined the discharge process for providers and nurses and provided consistent, evidence-based disease-specific discharge medications and instructions for patients and caregivers.
Health Literate mini-after care instructions
A local survey evaluating caregiver reasons for unscheduled 72-hour return helped inform the development of over 50 brief after-care instructions or mini-after care instructions (mini-ACIs).9 These half-page mini-ACIs emphasized 4 essential, “need to know,” components of discharge: (1) diagnosis, (2) duration of illness, (3) home care, and (4) return precautions (Fig. 3). All mini-ACIs underwent review by a discharge committee composed of pediatric emergency medicine physicians, pediatricians, advanced practice providers, and nurses to ensure that content was standardized and evidence-based. Also, a health literacy expert reviewed the mini-ACIs to ensure that they met an 8th grade or lower literacy level, which is acceptable for the network of ED/UC’s patient population.
In collaboration with a health literacy expert, the team developed a curriculum focused on the teach-back method to improve provider and nurse delivery of discharge instructions and caregiver comprehension assessment. Teach-back is an Agency for Healthcare Research and Quality approved method to verify patient understanding of teachings by asking patients to verbalize what they need to know or do for their health.17 Prior studies demonstrate that the teach-back method can improve comprehension of instructions and compliance with home care and medication management.18,19 The team administered a pre-intervention assessment to providers and nurses to determine their baseline level of knowledge and utilization of the teach-back method. ED/UC administrators then required both nurses and providers to complete a computerized teach-back education module and a follow-up in-person skills check-off performed by a nurse champion. The team placed discharge scripts (Fig. 4) with prompts in ED/UC sites as a visual reminder to promote the use of teach-back during discharge.
Study of Discharge Interventions
The Specific, Measurable, Attainable, Relevant, and Time-bound aim of this QI initiative was to decrease the unscheduled 72-hour return rate to a network of pediatric ED/UCs by 15%, from 3.5% to 3.0%, over 1.5 years. The primary outcome measure was the weekly percentage of unplanned 72-hour return rate across all ED/UC sites. The secondary outcome measures included the percentage of staff utilizing mini-ACIs and the teach-back method, which were estimated by evaluating the proportion of encounters with checked mini-ACI and teach-back discharge buttons in the EHR. The balancing measure was the admission rate of patients returning for a 72-hour unscheduled return visit.
The team obtained 1 year of pre-implementation data (January 2014 to December 2014) as a baseline, which provided data points to define the baseline 72-hour return rates. The team implemented improvement interventions between December 2014 and September 2015: EHR workflow (December 2014), mini-ACIs (April 2015), and teach-back method (September 2015). Post-intervention outcomes included data from September 2015 to April 2016. The team evaluated results using an internally-created weekly report from the EHR, providing 72-hour return and admission rates across all ED/UC sites.
The team created a statistical process control (SPC) chart to trend 72-hour return rates and to determine if interventions influenced observed outcomes. They used the average moving range method to calculate SD and control. Eight consecutive data points above or below the mean centerline specified special cause variation. The team then shifted centerlines at the point of special cause. They utilized chi-square tests to compare 72-hour return rates and admission rates pre- and post-interventions, and set the significance level at α = 0.05. The team performed statistical testing and created control charts using Minitab Statistical Software Version 17 (State College, PA).
During the study period (December 2014 to April 2016), the ED/UC network evaluated and discharged 219,196 patients; 123,275 (56.2%) were evaluated at an ED site and 95,921 at an UC site. At the time of this study, the baseline 72-hour return rate was 3.5% before any interventions (January 2014 to December 2014). The first intervention implemented was the standardization of the EHR discharge workflow in December 2014. Implementation of the EHR discharge workflow did not impact the 72-hour return rate, and no special cause variation was noted. During the next PDSA cycle, the study team incorporated the mini-ACIs and discharge smart sets into the discharge process. After the implementation of mini-ACIs, mini-ACI utilization was 63% (baseline 0%), and 8 consecutive points fell below the mean on the SPC chart demonstrating special cause variation at a return rate of 3.2% across the ED/UC network. The SPC chart was staged, demonstrating an 8.2% reduction in the 72-hour return rate (P < 0.01). Table 1 demonstrates the 72-hour return rates at ED and UC sites pre- and post-implementation of mini-ACIs. Special cause variation was achieved at UC but not ED sites (Fig. 5). During the last PDSA cycle, the team implemented the teach-back method (September 2015) with no additional improvement in 72-hour return rates. Utilization of teach-back method after implementation was 19% (baseline 0%). The balancing measure of admission rates of 72-hour return patients remained stable throughout the study and study interventions did not negatively impact rates: ED/UC network (27% pre-intervention versus 28% post-intervention, P = 0.23); ED sites (24% pre-intervention versus 23% post-intervention, P = 0.21), and UC sites (17% pre-intervention versus 14% post-intervention, P = 0.001). Overall, improvements to the ED/UC discharge process resulted in an estimated 600 preventable ED/UC visits annually throughout the system.
Table 1. -
Seventy-two-hour Return Rate Percentage Across ED/UC Locations
||Pre-mini ACI Intervention
||Post-mini ACI Intervention
|All sites (6 sites)
|ED only (2 sites)
|UC only (4 sites)
This QI initiative demonstrated a significant reduction in the unscheduled 72-hour return rate within a network of EDs and UCs using a multifaceted QI approach aimed at improving discharge processes. The use of health literate “mini-ACIs” proved most effective in reducing the 72-hour return rate. During the study period, the admission rate remained steady, suggesting that the patients’ illness severity was similar pre- and post-intervention and supported our intervention’s safety. An essential aspect of this QI initiative was the creation of a multidisciplinary team of stakeholders. The multidisciplinary team harnessed the experiences and expertise from a diverse group of providers across the multiple ED and UC sites. Importantly, this team provided individual site champions to obtain the support and engagement from a large group of over 250 providers and nurses. This project’s success supports the use of a multidisciplinary team and QI methodology to effectively reduce unscheduled 72-hour return visit rates at pediatric EDs/UCs.
This QI initiative elucidated interesting differences in the unscheduled 72-hour return visit rates at ED versus UC sites. Interestingly, the UC sites, which evaluate almost half of the patients in the health network, had a higher pre-intervention 72-hour return rate compared to ED sites. In this health network, UC sites are used as pediatric acute care settings where the academic tertiary care hospital cannot reach. Therefore, ED and UC sites are viewed and used similarly, and patients are not preferentially diverted to UCs over EDs. However, the UC sites have fewer resources (ie, lack of certain imaging modalities, labs, and pediatric emergency medicine specialists) than ED sites. The higher pre-intervention 72-hour return rate at UC sites may reflect the need for additional resources in these patients who returned for care. Also, a more significant reduction in 72-hour return rates was seen at UC compared to ED sites, and likely drove the reduction seen throughout the entire network. In a prior evaluation of caregivers of patients returning to the ED for a 72-hour return, the study team demonstrated that UC site was associated with complete delivery of discharge instructions (odds ratio 2.0, confidence interval 1.4–2.9).9 In the ED/UC network, the most substantial difference in discharge procedures is the health care provider who administers them: nurses discharge most patients at UC sites compared with ED sites, where many ED providers administer discharge instructions. Thus, it is possible that nurses were more effective at administering discharge instructions and adherent to interventions throughout the QI initiative. Of note, at the end of the study period, the 72-hour return rate at all sites began to increase towards the pre-intervention baseline, and special cause variation was noted above the mean. At this time, there was a transition in the QI team’s leadership resulting in a lapse in monthly notifications of 72-hour return rates to network providers/staff and monitoring of outcomes and process measures. Thus, the success and sustainability of the initiative relied heavily on the preservation of the QI team.
The multidisciplinary team determined that discharge represented a point-of-care where focused interventions could increase the overall quality and safety of patient care by improving caregiver understanding of their child’s ED/UC visit and home care. In this study, the implementation of mini-ACIs proved most effective in reducing the 72-hour return visit rate, likely because a survey of caregivers presenting to this network of pediatric ED/UCs for a return visit informed the mini-ACIs directly. The survey demonstrated that caregivers rarely understand all elements of discharge, specifically anticipated duration of symptoms.9 Thus, caregivers may not understand the natural course of their child’s illness, leading to increased concerns over symptom persistence. Therefore, the team created mini-ACIs to focus on critical components of discharge, highlighting the anticipated duration of illness, and to be at an eighth-grade reading level. Interestingly, the mini-ACIs have since been crucial to relaying a consistent message from all providers regarding the most common diagnoses evaluated in the ED/UC setting. The network of pediatric ED/UC is composed of over 150 different providers and includes advanced practice providers, general pediatricians, and pediatric emergency medicine specialists. A consistent message at discharge is key to providing standardized, evidence-based care.
The teach-back method of assessing caregiver comprehension did not improve the 72-hour return rate as initially hypothesized. Ultimately, the teach-back method may have been useful if better implemented across the system of ED/UCs. Our implementation rate of 19% was meager. While providers felt they understood and could use teach-back to some extent during discharge, many providers found it awkward and time-consuming. Studies demonstrating that teach-back is effective took place at single ED sites, where a unique group of either trained nurses or research assistants performed teach-back, which limits the scalability and clinical implementation.18,19 Implementing teach-back across a network of ED/UC sites with over 250 clinicians and nurses proved difficult and might require thoughtful dissemination and implementation strategies. Ultimately, the team implemented a modified teach-back script with an emphasis on asking open-ended questions and was more freely accepted by staff. While the team did not see an additional decrease in revisit rate after implementing the teach-back methodology, teach-back may still be an effective way to ensure comprehension of discharge instructions.
The multidisciplinary team acknowledged that EHR discharge standardization would most likely not decrease the 72-hour return rate. Still, it was necessary to improve the discharge workflow and increase buy-in for future PDSA cycles. While many providers appreciated a single discharge navigator, “late adopters” were reluctant to change from their previously established practice and did not immediately use the new workflow. However, the implementation of discharge smart sets with recommended mini-ACIs, medications, and follow-up increased provider usage of the single discharge navigator.
This study is not without limitations. First, the study took place from 2014 to 2016; however, the discharge interventions are still timely and applicable in the ED/UC setting. Also, the study does not provide a full year of post-implementation data, which is important to demonstrate the sustainability of the interventions. Further, not all institutions have access to health literacy experts or pediatric subspecialists to guide specific discharge recommendations and ensure evidence-based, health literate instructions. However, interventions such as teach-back are publicly available resources, and mini-ACIs are adaptable to various settings, provider types, and personnel administering discharge instructions.
Using QI methodology, multidisciplinary improvement of discharge processes, specifically health literate discharge instructions, significantly decreased 72-hour unscheduled return rates across a network of pediatric EDs and UCs. By embracing health literacy principles and placing emphasis on quality discharge teaching with improved readability and comprehensibility, this project enhanced discharge culture with a focus on patient and caregiver needs. This model of focused aftercare instructions combined with teach-back has potential for patients in other contexts, such as discharge from inpatient and primary care settings.
The authors have no financial interest to declare in relation to the content of this article.
Assistance with the study: Shelby Chapman, Director of Patient-Family Experience, Children’s Hospital Colorado; Mark Mares, Clinical Analyst, Children’s Hospital Colorado; Catherine Orendac, MD, MS, Senior Instructor, University of Colorado School of Medicine, Section of Emergency Medicine, Department of Pediatrics; Marion Sills, MD, MPH, Professor of Pediatrics, University of Colorado School of Medicine, Section of Emergency Medicine, Department of Pediatrics; Mary Willnerd, BSN, RN, Clinical Nurse II, Children’s Hospital Colorado.
1. Alessandrini EA, Lavelle JM, Grenfell SM, et al. Return visits to a pediatric emergency department. Pediatr Emerg Care. 2004; 20:166–171.
2. LeDuc K, Rosebrook H, Rannie M, et al. Pediatric emergency department recidivism: demographic characteristics and diagnostic predictors. J Emerg Nurs. 2006; 32:131–138.
3. Zimmerman DR, McCarten-Gibbs KA, DeNoble DH, et al. Repeat pediatric visits to a general emergency department. Ann Emerg Med. 1996; 28:467–473.
4. Ali AB, Place R, Howell J, et al. Early pediatric emergency department return visits: a prospective patient-centric assessment. Clin Pediatr (Phila). 2012; 51:651–658.
5. Rui P, Okeyode T. National Ambulatory Medical Care Survey: 2015 State and National Summary Tables. Visits to physician offices, hospital outpatient departments, and hospital emergency departments, by age, sex, and race: United States, selected years 1995-2011. 2015. Available at http://www.cdc.gov/nchs/ahcd/ahcd_ products.htm
6. Sills MR, Macy ML, Kocher KE, et al. Return visit admissions may not indicate quality of emergency department care for children. Acad Emerg Med. 2018; 25:283–292.
7. Duseja R, Bardach NS, Lin GA, et al. Revisit rates and associated costs after an emergency department encounter: a multistate analysis. Ann Intern Med. 2015; 162:750–756.
8. Kuan WS, Mahadevan M. Emergency unscheduled returns: can we do better?Singapore Med J. 2009; 50:1068–1071.
9. Navanandan N, Schmidt SK, Cabrera N, et al. The caregiver perspective on unscheduled 72-hour return visits to pediatric acute care sites: a focus on discharge processes. Acad Pediatr. 2017; 17:755–761.
10. Engel KG, Heisler M, Smith DM, et al. Patient comprehension of emergency department care and instructions: are patients aware of when they do not understand?Ann Emerg Med. 2009; 53:454–461.e15.
11. Makaryus AN, Friedman EA. Patients’ understanding of their treatment plans and diagnosis at discharge. Mayo Clin Proc. 2005; 80:991–994.
12. Jolly BT, Scott JL, Sanford SM. Simplification of emergency department discharge instructions improves patient comprehension. Ann Emerg Med. 1995; 26:443–446.
13. Isaacman DJ, Purvis K, Gyuro J, et al. Standardized instructions: do they improve communication of discharge information from the emergency department?Pediatrics. 1992; 896 pt 21204–1208.
14. Smith L, Daughtrey H. Weaving the seamless web of care: an analysis of parents’ perceptions of their needs following discharge of their child from hospital. J Adv Nurs. 2000; 31:812–820.
15. Samuels-Kalow M, Rhodes K, Uspal J, et al. Unmet needs at the time of emergency department discharge. Acad Emerg Med. 2016; 23:279–287.
16. Samuels-Kalow ME, Stack AM, Porter SC. Effective discharge communication in the emergency department. Ann Emerg Med. 2012; 60:152–159.
17. DeWalt DA, Broucksou KA, Hawk V, et al. Developing and testing the health literacy universal precautions toolkit. Nurs Outlook. 2011; 59:85–94.
18. Slater BA, Huang Y, Dalawari P. The impact of teach-back method on retention of key domains of emergency department discharge instructions. J Emerg Med. 2017; 53:e59–e65.
19. Griffey RT, Shin N, Jones S, et al. The impact of teach-back on comprehension of discharge instructions and satisfaction among emergency patients with limited health literacy: a randomized, controlled study. J Commun Healthc. 2015; 8:10–21.