Myers, Jennifer S.; Jaipaul, C Komal; Kogan, Jennifer R.; Krekun, Susan; Bellini, Lisa M.; Shea, Judy A.
Communication is essential to the practice of medicine. As hospitalizations become more complex and the transfer of care between hospitalists and primary care physicians becomes more common,1 the need for timely, accurate information exchange at hospital discharge becomes essential.2 Poor communication at discharge may increase adverse events after discharge3 and negatively affect patient safety and resource utilization.4,5
The hospital discharge summary is the primary document used for recording and retrieving the details of a patient’s hospitalization. Previous studies of discharge summary audits have found inaccuracies in the description of clinical events, medications, and follow-up information.6–8 The omission of critical discharge summary information, such as discharge medications, principal diagnosis, and diagnostic results, has been described.6–8 Brevity has also been cited as a desired feature of a quality discharge summary.9
In teaching hospitals, trainees unfamiliar with the format and importance of discharge communication are often responsible for creating discharge summaries. Although recommended requirements for discharge summaries are available,10 formal instruction has not been described. We conducted this study to assess the effects of a discharge summary curriculum on the quality, organization, and brevity of interns’ discharge summaries. We hypothesized that an educational intervention would improve some or all of these summary elements and that the addition of individualized feedback would be superior to a lecture alone. In addition, we sought to understand interns’ attitudes about discharge summary skills at baseline and obtain feedback on the curriculum’s utility.
This study took place on the hospitalist service at the Hospital of the University of Pennsylvania, a 600-bed university teaching hospital, from October through May in two consecutive academic years (2003–2004 and 2004–2005). At our institution, interns dictate discharge summaries for patients with an inpatient stay of greater than 48 hours. During the study period, interns were randomized based on month of the year to one of three arms: a control arm, a curriculum alone arm where instruction on discharge summaries was presented in lecture format, or a curriculum plus feedback arm where an individualized feedback session on discharge summaries was performed in addition to the lecture. Interns rotating June through September were excluded from randomization because there would have been too few dictated summaries available for study inclusion. Intern participation was voluntary, and informed consent was obtained from all participants. Our Institutional Review Board approved the study.
The curriculum was integrated into four hospitalist teaching services. Each service is composed of a hospitalist attending, a resident, two interns, and a third-year medical student. During curriculum months, the hospitalists each presented a 30-minute lecture on discharge summary skills that reviewed the essential elements of discharge summaries, presented strategies for dictating succinct, readable summaries, and emphasized the importance of discharge summary clarity and organization in physician communication. Examples of poor and high quality summaries were reviewed. During curriculum plus feedback months, the hospitalists also conducted an individual feedback session for each intern at a later date in that month. During the feedback sessions, which lasted approximately 15 minutes, two summaries dictated by the intern postcurriculum were evaluated. Prior to the study, all hospitalists participated in a one-hour session where curriculum content was reviewed.
A minimum of seven discharge summaries per intern were randomly selected in the four months pre- and post-intervention. Discharge summary word count was recorded for each summary. Discharge summaries were individually reviewed and graded with a nine-item discharge summary evaluation tool (DSET). This instrument was created by the investigators to assess for the presence of discharge summary elements required by the Joint Commission on the Accreditation of Hospital Organizations10 as well as to assess the overall quality of the summary. For the purposes of our study, a quality summary was one that was organized, succinct, internally consistent, easy to read, and contained all of the essential discharge summary components. Five items were scored yes/no (presence of discharge summary components, use of headings, presence and accuracy of the primary diagnosis, and secondary diagnoses and procedures). Four items were graded using a 7-point rating scale (quality of the history of present illness, quality of hospital course, summary length, and overall readability). Each summary was graded by one of four physician investigators who were blinded to intern group assignment. Prior to study initiation, these investigators reviewed and revised the DSET and collectively agreed upon grading criteria. Each investigator independently graded a random sample of 60 discharge summaries that were not included in the study. Interrater reliability was excellent for the five yes/no items (kappa = 0.61–0.92) and fair for the four graded items (kappa = 0.32–0.51).
For each DSET item, within group t-tests were computed to assess overall change, followed by analysis of covariance (ANCOVA) to compare differences among groups controlling for baseline performance. Because group sizes and therefore power were low, effect sizes among the three groups were calculated based on adjusted means.11 They are interpreted using the conventional definition of 0.20, 0.50, and 0.80 to represent small, medium, and large effects, respectively.12
A 6-item survey instrument was designed to assess interns’ attitudes about discharge summary skills. One hundred and eight interns were surveyed prior to study commencement to assess baseline attitudes. Intervention arm interns were resurveyed within 30 days of curriculum exposure. The percentage of interns agreeing with the survey statements post-intervention was compared with baseline values using Z-scores.
All interns (n = 108) agreed to participate in the study. Fifty-nine interns rotated on the hospitalist service between October and May of the study years. Of 59 interns randomized, 52 (control = 16, curriculum = 19, curriculum + feedback = 17) had a sufficient number of dictated discharge summaries pre- and post-intervention to be included.
Discharge summary quality improved within both intervention groups (see Table 1). Only one item showed significant improvement from pre to posttest (hospital course, p = .03) in the control group. For the curriculum only group, there were significant improvements for all items on the DSET with the exception of secondary diagnoses (p values from <.0001 to .04). As shown in the table, the curriculum plus feedback group showed improvements in five of the nine components. Several effect sizes in the curriculum and curriculum plus feedback group were moderate to large.
When post-intervention means were compared among groups, the ANCOVA results showed only two significant between group differences for DSET items: use of headings (p = .0002) and accuracy of procedures (p = .01). However, most group means were highest in the curriculum plus feedback group, second highest in the curriculum only group, and lowest in the control group showing incremental positive effects from the intervention (see Table 2). The effect sizes comparing pairs of groups are shown in Table 2. When comparing the control to the curriculum only group (Group 2–1), almost all effect sizes were small except for the better use of headings (ES = .68), procedures (ES = .68), and HPI (ES = .48) in the curriculum group. However, when the control group was compared to the curriculum plus feedback group (Group 3–1), there were consistent differences favoring the latter, and some effects were sizable: summary components (ES = .68), use of headings (ES = 1.98), accuracy of procedures (ES = .91), and readability (ES = .74). Comparisons between the curriculum only and the curriculum plus feedback group (Group 3–2), with one exception, were consistently positive and of moderate to large magnitude. Overall, the average effect size was small for the control to curriculum only group comparisons (.26), but large for the control to curriculum plus feedback comparisons (.70). Moreover, the results suggest an incremental benefit for the curriculum plus feedback compared to the curriculum only (.36).
All interns (n = 108) completed the baseline attitude survey, and 38 intervention arm interns (70%) completed the follow-up attitude survey post-intervention. At baseline, seventy-two interns (66%) reported receiving no prior instruction on discharge summary skills, and many (63%) were interested in learning these skills. The majority (52%) believed they should continue to hold the responsibility for dictating discharge summaries.
When compared with aggregate baseline responses, intervention interns were more satisfied with their discharge summary quality (55% versus 19%, p < .0001). Differences in other opinions were in the expected direction but were not significant: those receiving either intervention were more likely to believe that dictating discharge summaries solidified their understanding of a patient’s hospital course (40% versus 31%, p = .16) and more often believed that it was useful to develop discharge summary skills (85% versus 74%, p = .08). The majority of intervention interns believed that they would dictate better summaries after the curriculum (61%) and that all interns should receive the curriculum (79%).
We found that a brief educational session designed to teach discharge summary skills improved the quality of interns’ discharge summaries. Individualized feedback provided additional improvements in certain aspects of discharge summary quality. Most interns had minimal previous exposure to discharge summary instruction and many were interested in improving these skills. Among those exposed to the curriculum, subjective improvements in discharge summary skills were described. We believe that this curriculum led to improvements in discharge summary quality because the importance and purpose of this task was emphasized to our interns during the teaching sessions. In addition, it provided the interns with a set of tools that they could use during future summary dictations. We believe that the feedback session had additional positive effects because interns were able to see their summaries, review their deficiencies with an attending, and discuss strategies for improvement.
The results of this study are important for several reasons. First, in most teaching hospitals, the task of dictating discharge summaries is delegated to trainees who have minimal instruction in discharge processes and therefore limited understanding of the importance of excellent discharge communication during transitions of care. Our study is the first to describe a curriculum that teaches discharge summary skills to trainees and demonstrates positive effects on discharge summary quality. Second, it is a curricular innovation that engages trainees to think about the discharge communication process and the system in which they practice. Hence, such a curriculum addresses systems-based practice and practice-based learning and improvement, two American College of Graduate Medical Education core competencies.13 Third, improvements in discharge summary quality may have important downstream effects for practitioners who often depend on the content and completeness of the summary to make future clinical decisions. Finally, our results highlight the additive role that individualized feedback has on performance improvement, confirming the importance of feedback in clinical skills development.14
There are several limitations. First, this study involved internal medicine interns at a single academic medical center, thereby limiting generalizability to other institutions, other residency programs, or upper-level residents who might be responsible for discharge summary dictation. Discharge summary content and structure, however, is largely similar across institutions and residency programs, so adapting the curriculum in these settings would be feasible. Second, the curriculum was implemented by hospitalist faculty who received training in discharge summary skills instruction and who were invested in improving discharge summary quality. Teaching was integrated into their educational and clinical responsibilities, and opportunities for review of interns’ discharge summaries on shared patients were readily available. Whether nonhospitalist faculty or faculty with minimal training in discharge summary skills could effectively convey and teach the importance of discharge summaries skills is unknown. Third, we designed the DSET instrument for this study. Though psychometrically it appears promising, absolute agreement for the 7-point rating items was lower than anticipated. Finally, we cannot comment on the accuracy of the clinical content in individual summaries as they were not compared with the hospital charts. The goal of the curriculum, however, was to improve discharge summary quality as measured by the organization, brevity, and inclusion of specific summary components, all items that could be assessed without access to the patient’s medical record.
As hospitalizations become shorter and more complex and as the discontinuity of care between hospitalists and primary care physicians grows, the quality of discharge summaries is elevated in importance. Given that trainees play a central role in the discharge process in teaching hospitals, there is value in improving their discharge summary skills. We believe that future research designed to teach and improve specific aspects of the discharge process will be welcomed into hospital quality improvement initiatives and residency education.
We would like to thank Sigrid Warrender and Khalilah Mays for their assistance with the discharge summary database, Cathy Lloyd, RN, for her assistance with survey distribution, and the Penn Hospital Care Physician Group for their instruction of the curriculum.
1 Wachter RM, Goldman L. The hospitalist movement 5 years later. JAMA. 2002;287:487–94.
2 Pantilat SZ, Lindenauer PK, Katz PP, Wachter RM. Primary care physician attitudes regarding communication with hospitalists. Am J Med. 2001;111:15S–20S.
3 Forster AJ, Murgg HJ, Peterson JF, Ghandi TK, Bates DW. The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003;138:160–7.
4 van Walraven C, Seth R, Austin PC, Laupacis A. Effect of discharge summary availability during post-discharge visits on hospital readmission. J Gen Intern Med. 2002;17:186–92.
5 Moore C, Wisnivesky J, Williams S, McGinn T. Medical errors related to discontinuity of care from an inpatient to an outpatient setting. J Gen Intern Med. 2003;18:646–51.
6 Wilson S, Ruscoe W, Chapman M, Miller R. General practitioner-hospital communications: a review of discharge summaries. J Qual Clin Practice. 2001;21:104–8.
7 Macaulay E, Cooper G, Engeset J, Naylor A. Prospective audit of discharge summary errors. Br J Surg. 1996;83:788–90.
8 van Walraven C, Weinberg AL. Quality assessment of a discharge summary system. CMAJ. 1995;152:1437–42.
9 van Walraven C, Rokosh E. What is necessary for high-quality discharge summaries. Am J Med Qual. 1999;14:160–9.
11 Colliver J. Call for greater emphasis on effect-size measures in published articles in Teaching and Learning Medicine. Teach Learn Med. 2002;14:206–210.
12 Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Lawrence Earlbaum Associates, 1988.
14 Ende J. Feedback in clinical medical education. JAMA. 1983;250:777–81.