Padmore, Jamie S.; Jaeger, Joseph MPH; Riesenberg, Lee Ann PhD, RN; Karpovich, Kelly P.; Rosenfeld, Joel C. MD, MEd; Patow, Carl A. MD
The report of the Institute of Medicine, To Err Is Human,1 released in November 1999, generated considerable attention to and interest in issues related to errors and safety in teaching hospitals. Traditionally, adverse events (errors) related to residents' behaviors were accepted as unavoidable outcomes of learning to practice medicine effectively.2 Residency training is inherently stressful, and residents' accounts of their daily clinical pressures3 are well known to medical educators and hospital administrators. Residency education should be a time in which young physicians can practice in a safe and supervised environment.
Although residents and their institutions have many common goals, in practice these goals often diverge regarding how much to emphasize academics and how much to emphasize patient care. Residents may place equal emphasis on academics and on patient care, but their ultimate priorities are clinical competence, graduation, and specialty board certification. In contrast, the hospital is much more patient-care-oriented and includes clinical quality and reduction of errors as top priorities.
By nature, residency training also is “transient,” and for some it is seen as simply a means to an end. Residents frequently matriculate to further training or practice at other institutions or sites. As a result, many residents, unlike hospital employees or medical staff, may not fully engage in hospital-specific initiatives and are often seen by themselves and others as “renters” rather than “owners” in the culture of the institution.
There is growing acceptance that systemic problems that are central to complex adaptive systems, as opposed to individuals' isolated performance, contribute more significantly to errors in health care.4 As a result, many individuals see the “problem” to be with the system and not with their role in the system, thus avoiding ownership in effective improvement. This premise is particularly relevant in graduate medical education (GME), where the residents commonly view themselves as unable to invoke meaningful system improvements. Wu et al5 examined how house officers learned from their own mistakes, but noted that the house officers failed to learn from the mistakes of others.
The purpose of our study was to conduct a systematic review of the literature focused on residents' attitudes and behaviors regarding medical errors in teaching hospitals, including a qualitative review of barriers and countermeasures related to residents' engagement in patient safety. We conducted this review in conjunction with the Alliance of Independent Academic Medical Centers' National Initiative: Improving Patient Care Through GME. This national initiative was a collaborative formed in 2007 that linked residency programs in 19 teaching hospitals across the United States in efforts to integrate academics and quality through projects coordinated at a national level.
The methodology that we, the review team, employed consisted of regular, substantive discussions about manuscript concept and design, such as key questions, inclusion and exclusion criteria, and search strategies. There were critical interchanges about intellectual content, editing, and consensus on the final manuscript. The specific subject, appropriate technique, and final presentation of this review are the product of a progressive, iterative, and qualitative process of refinement, described below.
The review team explored the literature on residents and error via electronic literature databases, consisting of Ovid MEDLINE, the Cumulative Index of Nursing and Allied Health Literature (CINAHL), and the Education Resource Information Center (ERIC).
The search was limited to the United States and European English-language articles published between January 1, 1988 and June 30, 2008. The following medical subject headings (MeSH) were used to search the databases: hospital or teaching hospital; graduate medical education or medical education; intern$, internship or residen$, residency or housestaff or house officer; these terms were also combined with medical error$ or clinical error$. Additional terms were perception, attitude, or response. The core search was limited using the terms NOT work hours, duty hours, fatigue; simulation; nursing; or medical students; or comment or editorial or letter or case report. All MeSH terms were exploded with the exception of hospital, teaching; 124 articles were initially identified. A title and abstract review was conducted by two of us (J.S.P. and J.J.), and 55 articles were identified for a full-text review, including a search of the references of all included articles. Relevant reference articles were reviewed for inclusion in the study. In addition, we consulted experts in the field of GME for additional articles. Relevant studies were archived using EndNote X1 (Thomson Corporation, Carlsbad, California).
Criteria were established to identify English-language articles of research conducted in the United States, Canada, and/or Europe. Studies were included if they specifically described data collected from residents regarding their attitudes, behaviors, and subsequent practices with regard to medical errors and error reduction and/or factors contributing to residents' role in error reduction or related systems improvement. Studies were excluded if they (1) did not occur in the United States, Canada, and/or Europe, (2) did not assess residents' responses to error, (3) did not analyze data on residents' attitudes and behaviors regarding their personal role in reducing error, (4) did not describe residents' perceptions of the clinical systems contributing to error, or (5) were editorials or single-case studies.
Articles also were excluded if they focused on duty hours or fatigue; simulation in relation to error; rate of error, disclosure of error, role of morbidity and mortality conference in error; and programs to reduce error.
Title and abstract review
All citations produced from the above MeSH and reference searches were independently reviewed by two of us (J.S.P. and J.J.). In the first of three phases, the titles and abstracts of the 124 citations were reviewed for eligibility and categorized as either not applicable or applicable. If disagreement occurred regarding a citation, the full text was reviewed for further discussion. A total of 55 articles were considered “applicable,” and these articles' full text was reviewed for eligibility (J.S.P. and J.J.). Nineteen articles were identified as meeting all eligibility criteria.
Data were extracted independently from the 19 identified articles by two of us (J.S.P. and J.J.) using a standardized form with preidentified data fields. Variations in abstracted data were reviewed and discussed by both of us until a consensus was reached. Extracted information included the date of the project, type of institution, clinical specialty of the program(s), study type, sample size, response rate, and strengths, limitations, and results of the research. Barriers and countermeasures were identified from each article and listed in a matrix format. Because there was not a priori classification (either based on theory or existing in the literature), two of us (J.J. and J.S.P.) identified general themes and, through an iterative process, agreed on category labels. All barriers and countermeasures were then placed in a category or subcategory.
Quality scoring system
Downs and Black6 created a valid and reliable checklist designed to assess both experimental and observational studies. Two systematic reviews7,8 of published systems (scales and checklists) designed to assess study quality have ranked the scale developed by Downs and Black as one of the best. Both of these systematic reviews went on to suggest that some modifications might be useful depending on the specific topic and study designs. Therefore, three of us (J.J., J.S.P., L.A.R.) used a descriptive-analysis quality scoring method described in a related study.9 Using four of the original items and eight modified items yielded scores ranging from 1.0 to 16.0, with 16.0 being the highest possible score (see Chart 1). In addition, the sample size scoring was modified to include only residents, as that is the focus of this review. This quality assessment form contained two items related to study type and sample size, five items related to reporting, and five items related to internal validity.
Quality scores were independently obtained from two reviewers (J.S.P. and J.J.). The interrater reliability was assessed for all identified research studies. A third reviewer (L.A.R.) moderated the discussion to yield a final quality score.
Results and Discussion
Of the 19 articles that we identified that described residents' responses to medical errors within the institutional context,5,10–27 all but two5,25 were published since 2003 (see the Appendix). The majority of the studies fell within three major study design categories, with the majority being either single-group cross-sectional or qualitative. The 12 cross-sectional studies included 10 baseline survey assessments, 1 questionnaire, and 1 combined survey–interview. The 5 observational studies all used interviews (4 used transcript and 1 used inductive analysis). The third category was cohort and single-group pre- and posttest. One study28 used an educational intervention with a pre- and posttest, and one was a prospective longitudinal cohort study.18
Content analysis yielded seven personal, six environmental, and four system barriers, as well as three environmental and five system countermeasures (see Table 1). The most common barriers were environmental, with potential punishment/career impact and fear of legal ramifications being the most common in this category. Residents' responsibility or involvement, and residents' inexperience, along with inadequate systems, were the most common personal and system barriers, respectively.
Quality assessment scores ranged from 5.5 to 12.5. Three studies obtained a score of ≤8.0, 5 studies obtained scores between 8.5 and 10.5, and 11 studies achieved scores between 11.0 and 12.5. Overall agreement between the main reviewers was 94.7% and Cohen's kappa was r = 0.88, P < .001.
In the 19 articles that we identified, most investigators studied primary care programs (i.e., internal medicine, family practice, pediatrics), whereas several studies were institution-wide or surgically oriented. In most studies, the response rate was >70% and had a total number of participants of >100. However, several limitations exist. Most studies were limited to one program or institution, limiting the ability to generalize the results. The vast majority of studies were baseline assessments consisting of surveys, questionnaires, or interviews. In fact, there was only one cohort study18 and one study with an intervention.29 This limits our ability to speak with confidence with regard to the effectiveness of the countermeasures identified.
Recall of one's own error is both a strength and a limitation in these studies. Potential sources of bias are prevalent in recall of human error. Because many of these studies were based on residents' recall of their own clinical errors, it is difficult to determine the accuracy of the events as described. Whereas many of these interviews were iteratively developed and piloted for reliability, the surveys, questionnaires, and interview methodology were not consistent across studies.
Most studies were limited to short-term evaluations and did not address change over time. In general, the literature on this subject is relatively immature, as all but two of the studies that met our criteria were published between 2005 and 2008. Further support of this characterization is found in the fact that no study achieved or even approached the maximum score (16.0) in our quality scoring system. Whereas 11 studies clustered in the range of 11.0 to 12.5, even these studies reflected the lack of sophisticated study designs that would be expected in more robust literature.
There are numerous barriers identified in the literature, resulting in residents' reluctance to engage in institutional error identification and/or reduction. Although several countermeasures have been promulgated, the literature reveals scant evidence of their effectiveness.
Barriers fell within three interrelated categories: personal, environmental, and systemic. Environmental barriers were identified most frequently. Not surprisingly, residents' concern for the impact on their careers and reputations, including future punishment, seems to be a major barrier in identifying and responding appropriately to a medical error. A related concern, legal/malpractice issues, was mentioned to a lesser extent. The most important personal barriers included personal involvement and potential consequences of an error (including patient response), both of which pose emotional hurdles to responding appropriately. Among system barriers, the most prevalent were residents' inexperience and inefficient reporting processes. Finally, lack of time was only mentioned once, an unexpected finding, given the multiple demands placed on residents.
By far the most commonly cited countermeasure was environmental, which was described as “change organizational culture.” Authors suggested
* fostering appropriate attitudes and beliefs
* addressing malpractice fear
* addressing personal, environmental, and systemic barriers (for countermeasures to be effective)
* developing skills, which must be taught, practiced, and evaluated (to respond appropriately to health care errors)
* incorporating relevant instruction in residency programs
* assessing residents' competency in identifying and responding to errors
Numerous approaches to introducing strategies to promote patient safety and engendering a feeling of ownership have been suggested, including the development and implementation of a formal curriculum that might include guidelines, regular conferences, simulation, and evaluation, among other elements. Several authors suggested engaging the entire care team as part of an interdisciplinary role-modeling process, helping to share the burden of dealing with a medical error. It is worth noting that most authors, even when discussing systems, did not focus on the mechanics of the reporting process. Rather, they focused on broader issues such as the residents' role within the system at large.
Personal fear of retribution, ranging from academic failure to professional retaliation, was frequently mentioned as a factor contributing to residents' hesitancy to be leaders, or even to be involved, in various efforts to identify and/or reduce clinical error. Although most would agree that the culture of medicine has evolved significantly over the past several decades to be much more inclusive and multidisciplinary in terms of patient care and management, the clinical hierarchy of academic medicine often gives residents the impression that having questions and admitting unknowns are taboo. Most health care organizations have not adopted a “just culture”28 approach to reporting errors. Institutional reactions to reports of errors are often punitive, personal, and not systems based.
Despite the divergence in emphasis between trainees (principally academic-focused, such as acquiring necessary knowledge and skills, and career-focused) and the hospital/institution (principally mission-focused, maintaining financial viability, improving quality and safety), there also are many goals that are common between the two stakeholders. One such goal is the provision of high-quality patient care. However, the divergence in focus, as well as the transient nature of residency, separates the residents from the other hospital staff culturally. GME leaders, program directors, and teaching faculty may be the key to bridging this gap by working to align hospital quality and safety goals and initiatives into the curriculum of the residency programs. As the requirements of the Accreditation Council for Graduate Medical Education continue to evolve and become more focused on outcomes, the integration of academics and quality is likely to be the essential link needed to improve teaching, learning, and clinical and educational outcomes.
Although the current literature has its limitations, as noted above, a common theme is that the teaching faculty seem to recognize the importance of preventing patient-care errors and the potential for education to help reach that goal; however, they frequently struggle with identifying meaningful ways to incorporate patient safety into GME. Institutional leadership should recognize that residents have a unique perspective, based in part on their experiences, which span different departments within a single institution (e.g., emergency department, outpatient clinic, inpatient units) and external rotations in other hospitals. These distinctive experiences and fresh perspectives could prove to be a fertile source for shaping an institution's approach and response to patient safety. Further, mining these unique experiences may also be a way to meaningfully engage residents in the patient safety process, allowing for opportunities to enhance their sense of organizational “ownership.” Doing so may help address the apparent gap between physicians' attitudes toward and actual behaviors in response to error.
Strengths and limitations
There are several limitations of this study, some of which are inherent in attempting to summarize diverse studies that have vastly different aims and methodologies. There is not yet a common language surrounding the subjects of errors and safety as they relate to GME. The methods used here, including explicit search terms, iterative processes among experts, and establishing a transparent and seemingly reliable scoring system, are well-established approaches. However, the lack of an accepted framework for understanding, combined with the fact that many studies used iterative processes and not externally validated methods, means that we, the reviewers of the literature, may have relied too much on our subjective interpretations of articles.
Given the emerging nature of the field, relevant work may not have been submitted for publication in a peer-reviewed journal or, even if submitted, accepted. It is difficult to tell what effect, if any, this type of publication bias, along with the generally accepted publication bias (where studies with negative findings have a lower likelihood of being published), may have had. The literature searches, while designed to be comprehensive and reproducible, may not have contained all relevant terms; searching the references and asking experts helped mitigate against potential misses, but it cannot eliminate the possibility entirely. Further, because works such as editorials and case reports were excluded, potentially informative articles may be available but beyond the scope of the present study.
The scoring system, although based on a previously validated measure and now presented in two separate applications,9 is new and requires additional testing. A Cohen's kappa of 0.88 (P < .001) is a robust indicator of high interrater agreement, because it takes into account the role of chance and especially because κ is considered by many to be a conservative statistic (or even an underestimate of agreement). Another potential limitation of the system may be that although the reviewers were independent, the studies may not have been, in that multiple reports representing different aims from the same study (e.g., Kaldjian,13 Kaldjian et al,14 and possibly others) were included. This may have had the effect of increasing the agreement between reviewers, but the most likely effect would be to decrease the range of scores.
Future research should extend beyond university-based settings to include independent academic medical centers. Whereas primary care has been predominant in studies thus far, surgical-based specialties should be included in future studies. As noted in the literature,30,31 and consistent with the findings of this study, more sophisticated study designs should be employed to improve overall quality and consistency of the literature. Further, published reports of performance improvement efforts as they relate to residents and patient safety would benefit from using the “Standards for QUality Improvement Reporting Excellence” (SQUIRE) guidelines.29,32
Finally, future research should address the correlation between residents' engagement in the culture of an institution and their involvement in clinical quality improvement efforts and error reduction. Research should be conducted to determine why some hospital cultures create an environment that is as inclusive of residents as it is of nonresident employees/staff, whereas other hospitals perpetuate a feeling of detachment or transience.
Many thanks to Layla Heimlich, MLS, librarian at the Washington Hospital Center, MedStar Health, for assistance with article searching and retrieval; Frederic Pachman, MLS, AHIP, director of the Frank J. Altschul Medical Library at Monmouth Medical Center, for assistance with article searching and retrieval; Mr. Arjun Verma, for assistance with extraction of data; and Ms. Judith Reed for copyediting. Finally, to the Alliance of Independent Academic Medical Centers for the support and infrastructure of the National Initiative: Improving Patient Care Through GME.
1Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999.
2Leave L. Error in medicine. JAMA. 1994;272:1851–1857.
3Gaba DM, Howard SK. Fatigue among clinicians and the safety of patients. N Engl J Med. 2002;347:1249–1255.
4Reason J. Managing the Risks of Organizational Accidents. Aldershot, UK: Ashgate Publishing; 1997.
5Wu AW, Folkman S, McPhee SJ, Lo B. Do house officers learn from their mistakes? JAMA. 1991;265:2089–2094.
6Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality of both randomized and non-randomized studies of health care interventions. J Epidemiol Community Health. 1998;52:377–384.
9Riesenberg LA, Leitzsch J, Massucci JL, et al. Residents' and attending physicians' handoffs: A systematic review of the literature. Acad Med. 2009;84:1775–1787.
10Coyle YM, Mercer SQ, Murphy-Cullen CL, Schneider GW, Hynan LS. Effectiveness of a graduate medical education program for improving medical event reporting attitude and behavior. Qual Saf Health Care. 2005;14:383–388.
11Garbutt J, Brownstein DR, Klein EJ, et al. Reporting and disclosing medical errors: Pediatricians' attitudes and behaviors. Arch Pediatr Adolesc Med. 2007;161:179–185.
12Hobgood C, Hevia A, Tamayo-Sarver JH, Weiner B, Riviello R. The influence of the causes and contexts of medical errors on emergency medicine residents' responses to their errors: An exploration. Acad Med. 2005;80:758–764.
13Jagsi R, Kitch BT, Weinstein DF, Campbell EG, Hutter M, Weissman JS. Residents report on adverse events and their causes. Arch Intern Med. 2005;165:2607–2613.
14Kaldjian LC, Jones EW, Wu BJ, Forman-Hoffman VL, Levi BH, Rosenthal GE. Disclosing medical errors to patients: Attitudes and practices of physicians and trainees. J Gen Intern Med. 2007;22:988–996.
15Kaldjian LC, Jones EW, Wu BJ, Forman-Hoffman VL, Levi BH, Rosenthal GE. Reporting medical errors to improve patient safety: A survey of physicians in teaching hospitals. Arch Intern Med. 2008;168:40–46.
16Menachemi N, Shewchuk RM, O'Connor SJ, Berner ES, Allison JJ. Perceptions of medical errors by internal medicine residents: Development and validation of a new scale. Qual Manag Health Care. 2005;14:144–154.
17Muller D, Ornstein K. Perceptions of and attitudes towards medical errors among medical trainees. Med Educ. 2007;41:645–652.
18Sorokin R, Riggio JM, Hwang C. Attitudes about patient safety: A survey of physicians-in-training. Am J Med Qual. 2005;20:70–77.
19Vohra PD, Johnson JK, Daugherty CK, Wen M, Barach P. Housestaff and medical student attitudes toward medical errors and adverse events. Jt Comm J Qual Patient Saf. 2007;33:493–501.
20White AA, Gallagher TH, Krauss MJ, et al. The attitudes and experiences of trainees regarding disclosing medical errors to patients. Acad Med. 2008;83:250–256.
21Wild D, Bradley EH. The gap between nurses and residents in a community hospital's error-reporting system. Jt Comm J Qual Patient Saf. 2005;31:13–20.
22West CP, Huschka MM, Novotny PJ, et al. Association of perceived medical errors with resident distress and empathy: A prospective longitudinal study. JAMA. 2006;296:1071–1078.
23Engel KG, Rosenthal M, Sutcliffe KM. Residents' responses to medical error: Coping, learning, and change. Acad Med. 2006;81:86–93.
24Fischer MA, Mazor KM, Baril J, Alper E, DeMarco D, Pugnaire M. Learning from mistakes. Factors that influence how students and residents learn from medical errors. J Gen Intern Med. 2006;21:419–423.
25Mizrahi T. Managing medical mistakes: Ideology, insularity and accountability among internists-in-training. Soc Sci Med. 1984;19:135–146.
26Schenkel SM, Khare RK, Rosenthal MM, Sutcliffe KM, Lewton EL. Resident perceptions of medical errors in the emergency department. Acad Emerg Med. 2003;10:1318–1324.
27Sutcliffe KM, Lewton E, Rosenthal MM. Communication failures: An insidious contributor to medical mishaps. Acad Med. 2004;79:186–194.
28Dekker S. Just Culture: Balancing Safety and Accountability. Burlington, Vt: Ashgate Publishing; 2007.
30Torgerson CJ. Educational research at the crossroads. Med Educ. 2002;36:1002–1003.
31Colliver JA. The research enterprise in medical education. Teach Learn Med. 2003;15:154–155.
32Stevens DP. Why new guidelines for reporting improvement research? And why now? Qual Saf Health Care. 2005;14:314.
Table. Appendix Stud...Image Tools
Table. Appendix, con...Image Tools
Table. Appendix, con...Image Tools
Table. Appendix, con...Image Tools