Secondary Logo

Reduced Resident “Code Blue” Experience in the Era of Quality Improvement: New Challenges in Physician Training

Mickelsen, Steven MD; McNeil, Rebecca PhD; Parikh, Pragnesh MD; Persoff, Jason MD

doi: 10.1097/ACM.0b013e318217e44e
Graduate Medical Education
Free

Purpose Emergency resuscitation or “code blue” is a clinical event through which responding medical residents gain experience and proficiency. A retooling of practice has occurred at academic medical centers since the emergence of quality improvement initiatives and resident duty hours limits. The authors investigated how these changes may impact code blue frequency and resident opportunities to gain clinical experience.

Method The authors conducted a single-center, retrospective (2002–2009) review of monthly code blue frequency. They compared code blue frequency with corresponding monthly first-year internal medicine resident call schedules (2002–2008 academic years). Using a Monte Carlo simulation they estimated annual code blue experience, and using Poisson regression, they estimated annual trends in resident code blue experience.

Results The authors detected a 41% overall reduction in code blue events between 2002 and 2008; code blue events decreased by 13% annually (P < .001). These trends persisted, even after accounting for hospital census fluctuations: Rates fell from approximately 12 code blue events/1,000 admissions in 2002 to 3.8 events/1,000 in 2008. Overall, the model of code blue frequency and resident call schedules shows a dramatic reduction in the predicted number of code blue experiences, falling from 29 events (empirical 95% CI 18–40) in academic year 2002 to 5 events (CI 1–9) in 2008.

Conclusions Physicians-in-training at one facility are seeing far fewer code blue events than their predecessors. Whether current numbers of in-hospital code blue events are sufficient to provide adequate experience without supplemental practice for trainees is unclear.

Dr. Mickelsen is a cardiology fellow, Department of Internal Medicine, Division of Cardiovascular Diseases, University of Iowa, Iowa City, Iowa.

Dr. McNeil is statistician, Department of Epidemiologic Research & Information Center, Durham VA Medical Center, Durham, North Carolina.

Dr. Parikh is a cardiology fellow, Department of Internal Medicine, Division of Cardiovascular Diseases, Mayo Clinic, Jacksonville, Florida.

Dr. Persoff is assistant professor of medicine, Department of Hospital Internal Medicine, Mayo Clinic, Jacksonville, Florida.

Correspondence should be addressed to Dr. Mickelsen, University of Iowa Medical Center, Department of Internal Medicine, Division of Cardiovascular Diseases, 200 Hawkins Drive E318-5 GH, Iowa City, IA 52242-1081; telephone: (319) 356-2585; fax: (319) 353-6343; e-mail: steven-mickelsen@uiowa.edu.

First published online April 20, 2011

Editor's Note: A commentary on this article appears on page 680.

Resident training programs are under pressure to provide adequate clinical experience in a rapidly changing hospital environment. The emergence of quality improvement (QI) initiatives has catalyzed one of the greatest retooling efforts in U.S. medicine.1,2 Conceivably, such a changing hospital environment can result in fewer opportunities for residents to perform procedures, manage medications, and/or gain experience with difficult clinical scenarios.3–9 In addition, recent resident duty hours changes may also reduce exposure to certain hospital events. Emergency resuscitation or “code blue” is one example of a clinical event through which many residents gain experience and, in turn, proficiency. The effects of the new duty hours restrictions and the evolving hospital environment on physicians-in-training are not well studied.

Thus, we examined trends in the number of opportunities first-year internal medicine residents had to participate in code blue events after the implementation of duty hours limits and during a period of QI-driven practice change.

Back to Top | Article Outline

Method

Overview

This is a single-center, retrospective analysis of code blue events over a seven-year period. We used code blue event frequency and resident schedules to generate a probabilistic model. We applied the model to predict the number of code blue events a first-year resident might experience at our program.

Back to Top | Article Outline

Setting

Our facility is a tertiary referral center with approximately 12,000 admissions annually and an internal medicine program for 33 residents. While on call, first-year residents carry a code blue pager and must—along with the senior resident who is on call—respond to all code blue alerts for the duration of the 24-hour call period. Prior to 2004, residents worked seven inpatient rotations, including 24-hour code-team duty every fourth day (Q4-day). This schedule changed with the introduction of the night float system in 2004, in part to accommodate the Accreditation Council of Graduate Medical Education (ACGME) 80-hour-per-week duty hours limitations. Since then, the first-year residents work five inpatient rotations including 24-hour code-team duty every sixth day (Q6-day). First-year residents also respond to code blue during intensive care unit (ICU) and emergency room (ER) rotations. At our facility, senior internal medicine residents (second- and third-year residents) run the code blue response with a pharmacist, nurse, and respiratory therapist, and the first-year residents participate in direct patient care. In many cases an attending physician is present to supervise. In their first year of training, residents develop skills in managing code blue through experience; in the second year of training they assume the principal role.

In this study, we define code blue as any event generating documentation in the medical record corresponding to emergency response in the hospital, ICU, and/or ER. These events are heterogeneous and represent a variety of acute presentations including stroke, respiratory arrest, cardiovascular collapse with pulseless electrical activity, and/or cardiovascular collapse with arrhythmias. Using our Utstein template database, we reviewed all the code blue events from 2002 to 2009 that occurred at our facility.10 We evaluated the data for location (i.e., department) of the code blue, in-hospital mortality, and incidence by calendar and academic years. We compared our single-center estimated incidence of code blue involving cardiopulmonary resuscitations per admission with national data using International Classification of Diseases codes associated with resuscitation procedures (i.e., 99.60, 99.63, 93.93). These national data derived from the Agency for Healthcare Research and Quality (AHRQ) online database.11

Back to Top | Article Outline

The model and statistical methods

We developed a computational model, based on our historic resident schedules and month-to-month incidence of code blue, to predict the annual frequency of first-year residents' experiences with code blue. At our facility all code blue team obligations correspond to the on-call schedule. Although monthly schedules were available for each first-year resident going back to 2002, specific information on code blue participation and/or specific call days was not. To calculate the number of times an individual resident may have participated in code blue events during a given in-hospital rotation month, we applied a Q4-day or Q6-day call schedule correspondingly.

We ran our Monte Carlo simulation separately for each month using a random distribution of known code blue events in that corresponding month. Specifically, we assumed that the number of code blue events would have a Poisson distribution (i.e., a distribution commonly used to describe the occurrence of events over time). When the resident was in an on-call period, the simulation randomly generated code blue events according to that distribution. For example, for the months when the resident was on ER rotation, the simulation randomly generated events based on the estimated rate of ER code blue events. Likewise, when the resident was on ICU rotation, the simulation randomly generated events based on the estimated rate of all facility code blue events, reflecting the requirement that the resident respond to all code blue alerts during that rotation. For the months when a resident was on an in-hospital rotation, the simulation generated events only on the on-call days, according to the Q4-day or Q6-day schedule. Finally, when the resident was on outpatient or elective rotations, he or she was not considered to be on call, and no events were generated. Thus, we did not account for any events in which residents may have responded to code blue outside of their code blue team obligations.

We generated the overall trends in yearly experience as the total of individual month-to-month results based on the historic corresponding resident schedule: six to eight in-hospital (Q4-day or Q6-day call schedules) rotations, one ER rotation, one ICU rotation, one outpatient rotation, and one to three elective rotations. The use of the Poisson distribution to simulate the number of code blue events allowed random variation in the number of events, creating a stochastic process. This permitted our Monte Carlo simulation— performed a large number of times—to mimic the variation that would occur over time in the natural hospital setting. We performed simulations a minimum of 2,000 times. Poisson regression was used to estimate the change in real and predicted event rates over the years considered.

We exported data from Excel to SAS Version 9.1 (SAS Institute Inc., Cary, North Carolina) for statistical analysis.

Back to Top | Article Outline

Results

Code blue events

Over the seven years studied, 973 code blue events occurred, and 78,900 patients were admitted. The number of yearly code blue events decreased by an estimated 13% annually (P < .001), and the overall number of code blue events decreased by 41% between calendar years 2002 and 2008 inclusive (see Figure 1). These trends persisted even after accounting for fluctuations in hospital census; code blue event rates fell from approximately 12 code blue events per 1,000 admissions in 2002 to 3.8 code blue events per 1,000 admissions in 2008. In contrast, data from AHRQ show that the national rates of patients receiving resuscitation increased significantly by 3.3% yearly from 1999 to 2007 (P < .001).11 Reduced in-hospital code blue incidence at our center was not accompanied by a significant change in survival to discharge (P = .17).

Figure 1

Figure 1

Back to Top | Article Outline

Resident experience with code blue

Overall, the model of code blue frequency and resident call schedules shows a drastic reduction in the predicted number of potential code blue episodes residents will experience, decreasing from 29 events in 2002 (empirical 95% confidence interval [CI] 18–40) to 5 events in 2008 (empirical 95% CI 1–9). This is effectively an 83% reduction in potential code blue events for our first-year residents. Resident scheduling changes in 2004 had the greatest single effect on resident code blue experience with a 44% reduction (P < .001) in predicted events in a single year. In fact, our stochastic modeling of code blue frequency and resident call schedules shows a drastic reduction in the number of probable code blue encounters occurring that year: Probable number of cases decreased from 27 (empirical 95% CI 17–37) in 2003 to 15 (empirical 95% CI 8–23) in 2004 (see Figure 2). Based on our model, future interns are unlikely to participate in no more than 5 code blue events per year. Indeed, if the frequency of code blue events continues to decrease, our model predicts that in 2009 we could expect that 2.4% of our first-year residents would not participate on a single code blue call and that in 2010 as many as 5% would not.

Figure 2

Figure 2

Back to Top | Article Outline

Discussion

Fewer code blue events

Over the period of study we observed a dramatic reduction in the number of code blue events at our facility. Although this is a welcome change, the scarcity of code blue events can pose a challenge to training programs relying on experience as their principal means of preparing physicians-in-training. Mastery of emergency resuscitation is no less important in an era of fewer code blue events, and as the frequency of code blue events dwindles, physicians-in-training may find that acquiring the specialized skill set needed for successful practice is difficult.12 In our own program we depend on the experience gained in the first year to prepare the resident to lead in the resuscitative efforts in subsequent years. Even well-trained personnel adhere to resuscitation guidelines inconsistently; infrequent experience will only worsen the problem.13–16

Back to Top | Article Outline

Impact on physician training

How much experience new physicians require to be able to successfully manage code blue events or apply advanced cardiac life support (ACLS) is not defined. In studies conducted at academic medical centers, where many residents gain the clinical experience they need to practice independently, the code blue incidence ranges from 7 to 12 per 1,000 admissions.17,18 In a survey of over 200 medical centers, the average hospital sees just over 50 events requiring application of ACLS annually.19 However, current code blue rates at our facility are substantially lower than both of these estimates. Also, the scheduling changes we implemented in 2004 to comply with the 2003 ACGME duty hours limitations20,21 resulted in approximately one-third fewer code blue team obligations for our first-year residents. The combined effect was a dramatic reduction in the total number of opportunities for first-year residents to participate in a code blue event—decreasing from more than 30 to fewer than 5 annually. Case in point, we recently had one resident complete the first year of training without participating in a single code blue event, and by our estimates, this will probably occur again. Even participating in five code blue events a year (i.e., the number predicted by our model) may not constitute adequate preparation for the independent management of emergency resuscitation.

Back to Top | Article Outline

QI initiatives

In-hospital code blue is the final common pathway for several complications identified in the Institute for Healthcare Improvement's 100,000 Lives Campaign.22–24 Like many institutions throughout the United States, our hospital has implemented QI initiatives that are directed at reducing in-hospital emergencies. Patient-directed QI efforts can potentially impact the number of code blue events in a number of ways. Clear identification of patients' wishes for resuscitation can help avoid unwanted code blue events.25 Systematic hospital practices directed at preventing pulmonary embolism, medication errors, and sepsis—as well as improvements in multidisciplinary communication—also potentially reduce the need for emergency resuscitation.26–33 Although controversial, early intervention by specialized medical and/or rapid response teams might further reduce the number of inpatient emergencies and improve mortality.34–37 In the last decade, our facility implemented numerous QI initiatives, which go through an iterative process of reassessment and modification. These practices have led to a high degree of compliance with guidelines aimed at improving advanced directives or “code status” documentation and reducing incidence of reported pulmonary embolism, line infections, medication errors, and unnecessary urinary catheter use. The 13% mean annual reduction in code blue frequency that we observed argues against the priority of a single, independent factor driving the change. The QI initiatives put into place over the study period clearly changed our hospital practice, and these changes were contemporaneous with the observed reductions in code blue frequency; however, further work would be needed to definitively connect QI practices and these observed reductions in code blue events.

Back to Top | Article Outline

General implications for residency programs

This study highlights how a rapidly evolving hospital environment can challenge historic paradigms of medical education. In this example, the incidence of previously common resident experiences with code blue became relatively rare over a remarkably short period of time. In the meantime, our approach to training residents in emergency resuscitation went unchanged. Strategies such as code blue simulation may begin to address the need for mastery-level training in the absence of real-world opportunities.16,38 However, simulation can be very expensive, and without clear evidence of improved patient outcomes, cost-conscious hospitals may be reluctant to adopt such practices.39–42

This study also underscores the value of using data to direct priorities in the training environment. We cannot assume that a resident entering any training program today will have the same opportunities to see, do, and teach medicine as their senior residents or junior faculty—much less, the senior faculty—had. The number of opportunities to evaluate and manage key medical scenarios during training is finite, so relying on luck to provide these key clinical events may be inappropriate. Therefore, more effort should be made to evaluate the impact and quality of key experiences in the context of a training program's learning objectives.

Mathematical modeling of resident experience can provide an efficient way to evaluate surpluses and deficits in an individual program's environment. A mathematical model can also predict the potential impact of proposed changes on future resident learning opportunities. We recognize that the skill set of in-hospital resuscitation is not necessarily applicable to all residency training programs, but program directors could adapt this kind of analysis to almost any experience-dependent training issue. In fact, the data presented here have provided us with an opportunity to adapt our curricula to meet residents' individual training needs. These data also essentially constitute—in and of themselves—a QI project, and certainly, resident experience with well-performed QI initiatives may be more relevant preparation for future clinical work than practice performing any single procedure.42,43 We can take lessons from our data and transform a historical perspective on the adequacy of resident training into a truly evidence-based endeavor.

Back to Top | Article Outline

Limitations

From the current analysis, we cannot establish a causal relationship between any single, patient-directed QI initiative and the reduced number of code blue events at our institution. We raise the possibility of the association because QI initiatives are driving many of the changes occurring in U.S. hospitals, including our own. The trend in reduced code blue events is clear, but a different study design would be needed to assess the impact of each QI initiative on code blue frequency. The reduction in code blue frequency could also possibly reflect progressive admission of less sick patients over the study period. We did not perform a comorbidity severity index analysis to control for this potential confounder44; however, over the study period, the annual number of hospital admissions increased minimally, whereas the length of stay and average age of patients did not change substantially.

Back to Top | Article Outline

Conclusions

Current physicians-in-training may have fewer opportunities to participate in code blue events than did their predecessors. Resident-directed QI initiatives, in the form of work hours restrictions, substantially reduced resident opportunity for code blue experience. Furthermore, we observed significant annual reduction in hospital code blue events over the last seven years. Recent hospital practice changes, driven largely by QI initiatives, may have contributed to this observed reduction in total code blue events. It is unclear whether current numbers of in-hospital code blue are sufficient to provide adequate experience without supplemental training for residents managing these events. Future studies will need to address the direct impact of QI on patient safety, resident experience, and outcomes.

Back to Top | Article Outline

Acknowledgments:

The authors are grateful to the anonymous reviewers of this manuscript for their insightful comments and efforts.

Back to Top | Article Outline

Funding/Support:

None.

Back to Top | Article Outline

Other disclosures:

Statistical work was completed while Rebecca McNeil was research associate at Mayo Clinic Biostatistics Unit, Jacksonville, Florida.

Back to Top | Article Outline

Ethical approval:

Not applicable.

Back to Top | Article Outline

Previous presentations:

This work was presented in part under a different title, “A paradox of quality improvements: Reduced code blue events and physician training,” at the Mayo Symposium on Quality Improvement, October 2009, Jacksonville, Florida.

Back to Top | Article Outline

References

1 Varkey P, Reller MK, Resar RK. Basics of quality improvement in health care. Mayo Clin Proc. 2007;82:735–739.
2 Daniel DM, Casey DE Jr, Levine JL, et al. Taking a unified approach to teaching and implementing quality improvements across multiple residency programs: The Atlantic Health experience. Acad Med. 2009;84:1788–1795. http://journals.lww.com/academicmedicine/Fulltext/2009/12000/Taking_a_Unified_Approach_to_Teaching_and.31.aspx. Accessed February 17, 2011.
3 Robinson MK, Mogensen KM, Grudinskas GF, Kohler S, Jacobs DO. Improved care and reduced costs for patients requiring peripherally inserted central catheters: The role of bedside ultrasound and a dedicated team. JPEN J Parenter Enteral Nutr. 2005;29:374–379.
4 Lucas BP, Asbury JK, Wang Y, et al. Impact of a bedside procedure service on general medicine inpatients: A firm-based trial. J Hosp Med. 2007;2:143–149.
5 O'Malley CW, Emanuele M, Halasyamani L, Amin AN; Society of Hospital Medicine Glycemic Control Task Force. Bridge over troubled waters: Safe and effective transitions of the inpatient with hyperglycemia. J Hosp Med. 2008;3(5 suppl):55–65.
6 Airee A, Guirguis AB, Mohammad RA. Clinical outcomes and pharmacists' acceptance of a community hospital's anticoagulation management service utilizing decentralized clinical staff pharmacists. Ann Pharmacother. 2009;43:621–628.
7 Freed GL, Dunham KM, Lamarand KE; Research Advisory Committee of the American Board of Pediatrics. Hospitalists' involvement in pediatrics training: Perspectives from pediatric residency program and clerkship directors. Acad Med. 2009;84:1617–1621. http://journals.lww.com/academicmedicine/Fulltext/2009/11000/Hospitalists__Involvement_in_Pediatrics_Training_.45.aspx. Accessed February 17, 2011.
8 Resnick AS, Todd BA, Mullen JL, Morris JB. How do surgical residents and non-physician practitioners play together in the sandbox? Curr Surg. 2006;63:155–164.
9 Eiser AR, Connaughton-Storey J. Experiential learning of systems-based practice: A hands-on experience for first-year medical residents. Acad Med. 2008;83:916–923. http://journals.lww.com/academicmedicine/Fulltext/2008/10000/Experiential_Learning_of_Systems_Based_Practice__A.7.aspx. Accessed February 17, 2011.
10 Jacobs I, Nadkarni V, Bahr J, et al. Cardiac arrest and cardiopulmonary resuscitation outcome reports: Update and simplification of the Utstein templates for resuscitation registries: A statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation. Circulation. 2004;110:3385–3397.
11 Agency for Healthcare Research and Quality Web site. www.ahrq.gov. Accessed February 17, 2011.
12 Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21:251–256.
13 Abella BS, Alvarado JP, Myklebust H, et al. Quality of cardiopulmonary resuscitation during in-hospital cardiac arrest. JAMA. 2005;293:305–310.
14 Grześkowiak M. The effects of teaching basic cardiopulmonary resuscitation—A comparison between first and sixth year medical students. Resuscitation. 2006;68:391–397.
15 Hayes CW, Rhee A, Detsky ME, Leblanc VR, Wax RS. Residents feel unprepared and unsupervised as leaders of cardiac arrest teams in teaching hospitals: A survey of internal medicine residents. Crit Care Med. 2007;35:1668–1672.
16 Spearpoint KG, Gruber PC, Brett SJ. Impact of the Immediate Life Support course on the incidence and outcome of in-hospital cardiac arrest calls: An observational study over 6 years. Resuscitation. 2009;80:638–643.
17 Okuda Y, Bryson EO, DeMaria S Jr, et al. The utility of simulation in medical education: What is the evidence? Mt Sinai J Med. 2009;76:330–343.
18 Peberdy MA, Ornato JP, Larkin GL, et al; National Registry of Cardiopulmonary Resuscitation Investigators. Survival from in-hospital cardiac arrest during nights and weekends. JAMA. 2008;299:785–792.
19 Chan PS, Nichol G, Krumholz HM, et al; American Heart Association National Registry of Cardiopulmonary Resuscitation (NRCPR) Investigators. Racial differences in survival after in-hospital cardiac arrest. JAMA. 2009;302:1195–1201.
20 Accreditation Council for Graduate Medical Education. ACGME duty hours. http://www.acgme.org/acWebsite/dutyHours/dh_index.asp. Accessed February 17, 2011.
21 Boex JR, Leahy PJ. Understanding residents' work: Moving beyond counting hours to assessing educational value. Acad Med. 2003;78:939–944. http://journals.lww.com/academicmedicine/Fulltext/2003/09000/Understanding_Residents__Work__Moving_Beyond.22.aspx. Accessed February 17, 2011.
22 Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; March 2001. http://www.nap.edu/html/quality_chasm/reportbrief.pdf. Accessed February 17, 2011.
23 Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100,000 Lives Campaign: Setting a goal and a deadline for improving health care quality. JAMA. 2006;295:324–327.
24 Leape L, Berwick D, Clancy C, et al. Transforming health care: A safety imperative. Qual Saf Health Care. 2009;18:424–428.
25 Chen J, Flabouris A, Bellomo R, et al. The Medical Emergency Team System and not-for-resuscitation orders: Results from the MERIT study. Resuscitation. 2008;79:391–397.
26 Scarvelis D, Anderson J, Davis L, et al. Hospital mortality due to pulmonary embolism and an evaluation of the usefulness of preventative interventions. Thromb Res. 2010;125:166–170.
27 Benjamin DM. Reducing medication errors and increasing patient safety: Case studies in clinical pharmacology. J Clin Pharmacol. 2003;43:768–783.
28 Cohen MM, Kimmel NL, Benage MK, et al. Medication safety program reduces adverse drug events in a community hospital. Qual Saf Health Care. 2005;14:169–174.
29 Aboelela SW, Stone PW, Larson EL. Effectiveness of bundled behavioural interventions to control healthcare-associated infections: A systematic review of the literature. J Hosp Infect. 2007;66:101–108.
30 Hoo WE, Muehlberg K, Ferraro RG, Jumaoas MC. Successes and lessons learned implementing the sepsis bundle. J Healthc Qual. 2009;31:9–14.
31 Alberts MJ, Hademenos G, Latchaw RE, et al. Recommendations for the establishment of primary stroke centers. Brain Attack Coalition. JAMA. 2000;283:3102–3109.
32 Peterson ED, Albert NM, Amin A, Patterson JH, Fonarow GC. Implementing critical pathways and a multidisciplinary team approach to cardiovascular disease management. Am J Cardiol. 2008;102:47G–56G.
33 Riesenberg LA, Leitzsch J, Little BW. Systematic review of handoff mnemonics literature. Am J Med Qual. 2009;24:196–204.
34 Jones D, Opdam H, Egi M, et al. Long-term effect of a medical emergency team on mortality in a teaching hospital. Resuscitation. 2007;74:235–241.
35 Konrad D, Jäderling G, Bell M, Granath F, Ekbom A, Martling CR. Reducing in-hospital cardiac arrests and hospital mortality by introducing a medical emergency team. Intensive Care Med. 2010;36:100–106.
36 Tee A, Calzavacca P, Licari E, Goldsmith D, Bellomo R. Bench-to-bedside review: The MET syndrome—The challenges of researching and adopting medical emergency teams. Crit Care. 2008;12:205.
37 Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M, Spertus JA. Hospital-wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300:2506–2513.
38 Langhan TS, Rigby IJ, Walker IW, Howes D, Donnon T, Lord JA. Simulation-based training in critical resuscitation procedures improves residents' competence. CJEM. 2009;11:535–539.
39 Paniagua D, Lopez-Jimenez F, Londoño JC, Mangione CM, Fleischmann K, Lamas GA. Outcome and cost-effectiveness of cardiopulmonary resuscitation after in-hospital cardiac arrest in octogenarians. Cardiology. 2002;97:6–11.
40 Riley RH, Grauze AM, Chinnery C, Horley RA, Trewhella NH. Three years of “CASMS”: The world's busiest medical simulation centre. Med J Aust. 2003;179:626–630.
41 McLaughlin S, Fitch MT, Goyal DG, et al. Simulation in graduate medical education 2008: A review for emergency medicine. Acad Emerg Med. 2008;15:1117–1129.
42 Patow CA, Karpovich K, Riesenberg LA, et al. Residents' engagement in quality improvement: A systematic review of the literature. Acad Med. 2009;84:1757–1764. http://journals.lww.com/academicmedicine/Fulltext/2009/12000/Residents__Engagement_in_Quality_Improvement__A.28.aspx. Accessed February 17, 2011.
43 Miller A, Archer J. Impact of workplace based assessment on doctors' education and performance: A systematic review. BMJ. 2010;341:c5064.
44 Li P, Kim MM, Doshi JA. Comparison of the performance of the CMS Hierarchical Condition Category (CMS-HCC) risk adjuster with the Charlson and Elixhauser comorbidity measures in predicting mortality. BMC Health Serv Res. 2010;10:245.
© 2011 Association of American Medical Colleges