Scott, David R. MD; Weimer, Melissa DO; English, Clea MPH; Shaker, Lynn RN, MPH; Ward, William MD; Choi, Dongseok PhD; Cedfeldt, Andrea MD; Girard, Donald MD
Dr. Scott is clinical instructor, Department of Medicine, Portland Veteran Affairs Medical Center and Oregon Health & Science University, Portland, Oregon.
Dr. Weimer is clinical instructor and general internal medicine fellow, Division of General Internal Medicine and Geriatrics, Department of Medicine, Oregon Health & Science University, Portland, Oregon.
Ms. English is division manager, Division of General Internal Medicine and Geriatrics, Oregon Health & Science University, Portland, Oregon.
Ms. Shaker is patient safety officer, Oregon Health & Science University, Portland, Oregon.
Dr. Ward is instructor of medicine, Department of Primary Care Internal Medicine, Mayo Clinic, Rochester, Minnesota.
Dr. Choi is associate professor of biostatistics, Department of Public Health and Preventive Medicine, Oregon Health & Science University, Portland, Oregon.
Dr. Cedfeldt is assistant dean, Department of Graduate Medical Education, Oregon Health & Science University, Portland, Oregon.
Dr. Girard is associate dean, Departments of Graduate and Continuing Medical Education, Oregon Health & Science University, Portland, Oregon.
Please see the end of this article for information about the authors.
Correspondence should be addressed to Dr. Scott, Department of Medicine, Oregon Health & Science University, 3181 SW Sam Jackson Park Road OP-30, Portland, OR 97239-3098; e-mail: firstname.lastname@example.org.
First published online April 20, 2011.
An important and growing national effort is under way to get academic clinicians and training programs to focus on quality improvement.1,2 Involving residents, in particular, is highly valuable to the successful planning and implementation of quality initiatives—as well as for instilling a sense of the importance of quality improvement processes in young physicians3,4—yet they are underrepresented in these efforts. A recent systematic review of the literature found “a great need for additional research on the effectiveness of residents' participation in [quality improvement] initiatives.”5
Despite widespread attempts to increase awareness surrounding identification and reporting of medical errors, the literature identifies only meager participation of attending and resident physicians in adverse-event reporting.6–9 The literature suggests that among hospitals with established reporting systems, physicians are responsible for submitting less than 3% of adverse events,7,10–12 leaving the vast majority of reports to nursing staff.6 The nominal involvement of physicians in reporting adverse events is especially troublesome given their unique positions at the forefront of health care delivery, which makes their insight highly valuable for system improvements.13
Barriers to physicians reporting adverse events include uncertainty surrounding the reporting mechanism, the types of events that should be reported, and who is responsible for reporting, as well as concerns about potential negative consequences of reporting.7,14,15 Although these barriers are remediable,7 limited data exist on successful strategies to overcome them.
Oregon Health & Science University (OHSU) is a large academic tertiary care facility where, historically, residents and faculty have been little involved in reporting adverse events; residents have submitted less than 2% of adverse-event reports. In this study, we assessed the impact a combination of a financial incentive and an educational campaign had on the reporting behavior of resident physicians.
We conducted a prospective cohort study involving all of OHSU's 680 residents and fellows (collectively referred to as “residents”). At the beginning of the 2008–2009 academic year, the House Officers' Association worked with the Division of Graduate Medical Education, the patient safety officer, and the hospital's chief financial officer to develop a quality initiative to increase residents' involvement in adverse-event reporting. The novel strategy included an educational campaign and a financial incentive that linked adverse-event reporting to performance-based resident retirement benefit eligibility. OHSU's institutional review board formally granted nonhuman subject exemption for this study.
After reviewing both OHSU's historical reporting data and those from the literature, we set a goal of having residents report 5% or more of all adverse events for the remainder of the 2008–2009 academic year. We chose this goal because, although it surpassed any level reported in the literature, we judged it to be reasonably achievable.
The economic incentive consisted of a retirement benefit in the amount of 1.5% of residents' annual salaries. Historically, OHSU has provided residents up to 3% of their annual salaries in retirement funds; half of this benefit (1.5% of annual salary) has depended on successful completion of a system-wide resident performance initiative.
The multifaceted educational campaign included e-mail notifications, audiovisual presentations, and one-on-one discussion. The House Officers' Association leadership sent monthly e-mails notifying housestaff of the initiative, its link to their retirement benefits, and the progress being made toward the 5% goal. They also worked with the Division of Graduate Medical Education and the patient safety officer to develop a brief (five-minute) PowerPoint presentation that was delivered during major conferences for several of the larger residency programs, including general surgery, pediatrics, internal medicine, emergency medicine, psychiatry, obstetrics–gynecology, and family medicine. The presentation addressed several of the barriers to adverse-event reporting, explaining how reports are submitted, which events merit reporting, and who holds responsibility for reporting various types of events. Considerable effort was put into dispelling myths of retaliation for reporting adverse events. The patient safety officer oversaw administration of these programs from September through October and represented several refresher programs from March through April. Finally, the residents serving on the House Officers' Association actively promoted the initiative through one-on-one communication with colleagues.
Residents reported adverse events, both before and after the initiative, via OHSU's electronic reporting system, Patient Safety Net, a real-time, Web-based event reporting system from University Health System Consortium, which is accessed on OHSU's intranet. Halfway through the initiative, in response to feedback from residents that the reporting system was difficult to access, OHSU placed a hotlink to the Patient Safety Net software on OHSU's electronic medical record system to facilitate access. Depending on complexity, a report takes approximately 10 to 20 minutes to enter into the system. The reporter must specify the patient's name and medical record number, the time and location of the event, the staff providing care at the time of the event, the event type (“medication error”; “adverse drug reaction”; “equipment or supply error”; “fall”; “error related to procedure or treatment”; “complication of procedure, treatment or test”; “transfusion reaction”; “behavioral issue”; “care coordination issue”; or “other”), a description of the event in the reporter's own words, the reporter's role in the event, a harm score (“no actual event”; “event, no harm”; “event, harm” (includes near-miss); or “event, death”), who was notified of the event (patient, family, nurse, etc.), recommendations for system improvement to prevent recurrence of the event, and whether the reporter would like feedback on the report.
We evaluated the adverse-event reporting data monthly beginning three months before implementing the initiative and continuing through the remainder of the academic year, ending June 1, 2009. The primary outcome measure was the percentage of all adverse-event reports across the institution that were submitted by residents. We also measured the relative change in reporting among different residency programs.
The Patient Safety Net system assigned one of three harm scores to each reported event— “harmful event,” “event, no harm,” and “near-miss”—based on user input and an algorithm modeled largely on the National Coordinating Council for Medication Error Reporting and Prevention Category Index 2000.16 We tracked the frequency of resident-reported events from each harm score category over the course of the initiative.
Following the study, we administered a nonincentivized online anonymous survey to all residents, which asked for their principal motivations for completing an adverse-event report. Respondents chose one or more applicable responses from among “Retirement benefit incentive,” “Education (e.g., learn about the process of submitting reports),” “Patient wellness,” and a free-text response option. The survey asked residents if they believed reporting had led to effective change by correcting system weaknesses that contributed to a reported event. It also asked them for perceptions regarding strengths and weaknesses of the reporting mechanism.
We performed statistical analyses via generalized linear model for Poisson distribution with log link function to estimate the effect of new initiatives and the educational campaign for reporting adverse events. Statistical computing was done with R language and environment developed by the R Development Core Team (www.r-project.org), Vienna, Austria. We defined a statistically significant P value as being less than .05.
During the study period of June 1, 2008 through May 31, 2009, 297 residents reported a total of 3,290 adverse events. The monthly average number of adverse events reported by residents increased by 5.5 times from 6 (1.6%) to 33 (9.0%) reports during the initiative (P < .001). In contrast, there was no significant change in the number of events reported by attending physicians (Figure 1). The number of reports by residents increased significantly in most specialties, with the exceptions of emergency medicine and anesthesia (Table 1).
Table 2 shows the changes in number of reported events by physicians-in-training grouped into three categories: near-miss, no harm, or harmful. The absolute number of reported events in each category increased significantly during the initiative. The proportion of near-miss event reports submitted by residents relative to those of the other two categories also increased significantly.
Of the 680 residents involved in the study, 108 (16%) responded to the survey. Of those, 76 (70%) reported having submitted an adverse-event report. Of the responders who had submitted an adverse event, 66 (87%) cited patient wellness as a motivation for reporting, 49 (64%) cited the financial retirement incentive, and 13 (17%) cited educational opportunity. Other reasons cited in the free-text option included a sense of moral obligation, a feeling of catharsis, and advice from an attending physician. Of the 76 responders who reported an event, 27 (35%) either somewhat or strongly agreed with the statement that their reports led to effective change by correcting system weaknesses that contributed to the adverse event, 56 (47%) did not know whether their reports had an effect, and 13 (17%) either somewhat or strongly disagreed with that statement.
As to their perceptions of the strengths and weaknesses of the reporting mechanism, 44 (83% of the 53 respondents who answered this question) found the Patient Safety Net software difficult or burdensome to use, whereas 4 (8%) residents thought it was easy to use. Eleven (21%) residents perceived the administration's failure to follow up or give feedback to be a major weakness of the reporting system, and 5 (9%) reported witnessing or being subject to retaliation for submitting a report, despite the confidential nature of the reporting system.
The novel approach of linking a financial incentive to residents' participation in quality improvement resulted in a striking increase in the number of resident-reported adverse events. During the initiative, 9% of all adverse-event reports were submitted by residents, far surpassing historical comparatives in this or any institution that has measured these data. Additionally, over the course of the initiative, residents increased their relative reporting of near-miss events (Table 2). We believe that identification and reporting of near-miss events requires proactive awareness of potentially harmful circumstances, a skill that residents may have developed through this initiative.
Although residents' reporting of adverse events significantly increased overall, this was not true of residents in anesthesiology and emergency medicine. We interviewed residents from these specialties and found several potential reasons for why this was so. Both groups already had robust continuous quality improvement processes for reporting and reviewing adverse events within their own departments. Both groups also, because of the fast pace of their work environments, frequently delegate entry of adverse-event reports to nurses or other support staff. During the study, anesthesiologists did not experience the targeted presentation on adverse-event reporting given to most of the residency programs; this, too, may have contributed to their lower rates of reporting. Finally, in contrast to all other listed specialties, neither group had active resident representatives in the House Officers' Association; this may have decreased the buy-in within these programs.
Although the economic incentive was a large factor leading to increased resident involvement, the survey suggested additional reasons for the initiative's success. In particular, the most commonly cited reason for reporting an adverse event was patient wellness. The multidimensional educational effort, too, likely allowed residents to overcome many of the barriers previously identified in the literature (Table 3).
Although a large proportion of residents (47%) surveyed felt that their reporting efforts led to effective change, room may exist for improvement in closing the loop between reported events and feedback regarding efforts to implement systems changes within our institution. Additionally, many comments within the free-text portion of the survey conveyed a sense of frustration with a perceived lack of response to reported events. These included such comments as “I have no idea if my report is going anywhere and whether anyone is reading it,” “I have heard of important changes being made but based on my experience, I have not seen it directly,” and “we were informed on initial action, but never the final implementation of change. I just assumed nothing happened.”
As a result of the nearly 300 adverse-event reports submitted by residents during the study period, the House Officers' Association and patient safety officer have developed quality improvement projects to address potential systems improvements. Many of the reports submitted by residents offered new insights into previously unnoticed or underemphasized systems issues. Specific systems that are being addressed include laboratory services as well as radiology and nursing communication (Table 4). The most common Patient Safety Net report from residents was that of lab orders being canceled without the provider being made aware due to the inability of the phlebotomist to locate the patient or access the patient who was at a procedure. Residents reported that such unexpected lab cancellations frequently led to delays in care. The House Officers' Association worked with the lab directors to create a new protocol in which lab cancellations could not occur without a physician's order. Since the initiative, an active House Officers' Association representative has regularly attended lab committee meetings to communicate important issues from a resident's perspective. The House Officers' Association also worked with the administration to help improve resident–nursing communication by issuing dedicated cross-cover pagers to on-call residents, making it easier for nurses and ancillary staff to identify and communicate questions or concerns to each patient's covering provider.
In addition to improving residents' involvement in adverse-event reporting, the retirement benefit incentive could be applied to a variety of resident-related quality improvement projects. This past year, for example, OHSU successfully implemented the same retirement benefit model to improve the rates at which residents give influenza and pneumococcal vaccinations to at-risk patients. A variety of other quality improvement measures could be identified and implemented in this manner with resident involvement.
The primary limitation of this study is that it was carried out in a single, large academic institution. Although it was not a multiinstitutional study, one would expect similar results at other academic institutions given the similarities of current adverse-event reporting rates by residents, as well as resident salaries and the nearly identical demography of residents in other Accreditation Council for Graduate Medical Education–accredited programs. Notably, the use of this approach relies on the existence or inception of a performance-based resident retirement benefit, which may not be feasible at other institutions.
A second limitation of this study is its relative lack of extended longitudinal follow-up. Despite the project's initial success in increasing residents' involvement in adverse-event reporting, it remains unclear whether the behavioral changes will be sustained. Further observations are needed to assess the durability of the initiative with regard to the behavior of both future residents within our institution and current residents following graduation.
Additionally, the limited response rate to our resident survey (16%) limited our insight into residents' feelings toward the initiative. A mandatory or incentivized survey of residents might provide a more thorough assessment of their perspectives.
Despite a nationwide effort to improve adverse-event reporting, there has been a general lack of engagement in such activities by physicians-in-training.6,7,9 Through a novel approach of integrating a retirement benefit and educational campaign, our institution has been able to dramatically increase residents' participation. The initiative has also resulted in a surprising increase in the relative frequency with which residents report near-miss events. Educational lectures and the act of reporting served to remediate previously described barriers to physician involvement in adverse-event reporting. Although not yet proven, the initiative is expected to have at least some degree of sustained impact on residents' behavior given that many of the identified barriers must only be overcome one time by each resident to have a durable impact. We recommend a similar approach to other academic institutions that hope to bolster residents' involvement in adverse-event reporting or other quality improvement projects.
The authors would like to acknowledge Oregon Health & Science University as well as the numerous resident and administrative leaders who participated in the implementation and success of this initiative. In particular, the authors would like to thank Christine Flores for her hard administrative work in helping to create this manuscript.
Salary support was obtained from Oregon Health & Science University.
This study was formally reviewed by the institutional review board at Oregon Health & Science University and was considered exempt on the basis of qualifying as nonhuman subject research.
The opinions expressed in this manuscript are those of the authors alone and do not necessarily reflect the leadership or others at Oregon Health & Science University.
English C, Shaker L, Ward W, Scott D, Magnusson R, Girard DE. A resident-driven quality improvement initiative to improve patient safety event reporting. Paper presented at: Annual Meeting of the Accreditation Council for Graduate Medical Education; March 2009; Grapevine, Texas.
4 Kaldjian LC, Forman-Hoffman VL, Jones EW, Wu BJ, Levi BH, Rosenthal GE. Do faculty and resident physicians discuss their medical errors? J Med Ethics. 2008;34:717–722.
6 Farley DO, Haviland A, Champagne S, et al. Adverse event reporting practices by US hospitals: Results of a national survey. Qual Saf Health Care. 2008;17:416–423.
7 Schectman JM, Plews-Ogan ML. Physician perception of hospital safety and barriers to incident reporting. Jt Comm J Qual Patient Saf. 2006;32:337–343.
8 Dollarhide AW, Rutledge T, Weinger MB, Dresselhaus TR. Use of a handheld computer application for voluntary medication event reporting by inpatient nurses and physicians. J Gen Intern Med. 2008;23:418–422.
9 Foster PN, Sidhu R, Gadhia DA, DeMusis M. Leveraging computerized sign-out to increase error reporting and addressing patient safety in graduate medical education. J Gen Intern Med. 2008;23:481–484.
10 Nguyen QT, Weinberg J, Hilborne LH. Physician event reporting: Training the next generation of physicians. In: Henriksen K, Battles JB, Marks ES, Lewin DI, eds. Advances in Patient Safety: From Research to Implementation. Vol 4. Rockville, Md: Agency for Healthcare Research and Quality; 2005: 353–360. http://www.ahrq.gov/downloads/pub/advances/vol4/Nguyen.pdf
. Accessed February 21, 2011.
11 Tuttle D, Holloway R, Baird T, Sheehan B, Skelton WK. Electronic reporting to improve patient safety. Qual Saf Health Care. 2004;13:281–286.
12 Milch CE, Salem DN, Pauker SG, Lundquist TG, Kumar S, Chen J. Voluntary electronic reporting of medical errors and adverse events. An analysis of 92,547 reports from 26 acute care hospitals. J Gen Intern Med. 2006;21:165–170.
13 Jagsi R, Kitch BT, Weinstein DF, Campbell EG, Hutter M, Weissman JS. Residents report on adverse events and their causes. Arch Intern Med. 2005;165:2607–2613.
14 Kaldjian LC, Jones EW, Wu BJ, Forman-Hoffman VL, Levi BH, Rosenthal GE. Reporting medical errors to improve patient safety: A survey of physicians in teaching hospitals. Arch Intern Med. 2008;168:40–46.