Peets, Adam D.; Boiteau, Paul J.E.; Doig, Christopher J.
Residency and fellowship training programs develop physicians by providing a structured and supervised clinical learning environment. Studies have found better patient outcomes in teaching hospitals compared to nonteaching hospitals.1–3 However, the specific effect that trainees have on patient outcomes is uncertain. Some studies have reported that trainees improve or make no difference in patient outcomes on the surgical and medical wards,4–8 whereas other studies suggest worse outcomes compared to care by an attending physician alone.9–12
In 1994, Pollack et al. reported increased mortality in pediatric intensive care units (ICUs) where junior residents were participating in patient care.12 This report generated concern in the lay press that ICUs staffed with resident physicians might pose an unacceptably high risk to patients. Since then, two further studies have assessed the effect of more experienced trainees in ICUs on patient outcomes.13,14 Both demonstrated decreased mortality rates in centers with critical care medicine fellowship training programs. However, the studies were unable to determine if the difference in outcome was due to the participation of the fellows or if the fellowship program at an institution was a surrogate marker for a center of excellence. To date, no study has assessed the effect of the fellows themselves, rather than the effect of the entire academic center, on patient outcome in the ICU.
The objective of this study was to evaluate the direct effect of critical care medicine fellows (CCMFs) on patient outcomes in academic adult medical/surgical ICUs.
A Royal College of Physicians and Surgeons of Canada accredited critical care medicine two-year fellowship training program has been active at the University of Calgary since 1990, enrolling one to three fellows per year. Royal College CCMF training programs require prior training in one of the following four “base” specialties: Internal Medicine, Anaesthesia, General Surgery or Emergency Medicine. An academic year is divided into 13 four-week blocks. The training program has 13 blocks of clinical critical care in one of three adult ICUs and 13 blocks of clinical experience in a discipline related to critical care (for example anaesthesia, nephrology or trauma) or research. When on ICU service, CCMFs are responsible for overseeing the daily patient management, and complete eight nights of call per block. Rotating junior residents from various specialties provide in-house coverage 24 hours a day. Critical care medicine (CCM) attending physicians are on service for seven consecutive days, and have overall responsibility for the patients admitted to the ICU. During blocks when a CCMF is on service, an increasing amount of responsibility is assumed by the CCMF as their training progresses. When there is no CCMF in the ICU, the patient care team consists of an attending physician and junior residents.
The Calgary Health Region (CHR) delivers treatment to the residents of the cities of Calgary and Airdrie and approximately 20 nearby small towns, villages, and hamlets (2003 population 1,122,521). The CHR has three adult teaching hospitals, each with a closed multidisciplinary medical/surgical ICU staffed by fully trained CCM attending physicians. The ICU’s include one 22-bed unit (also the regional trauma and neurosciences referral center), a 12-bed ICU (also the vascular surgery referral center), and a 10-bed unit. The CCM team admits all patients and is responsible for all patient care decisions. The study population consisted of all patients admitted to any of the three ICUs between July 1, 2000, and June 30, 2003.
This was a retrospective cohort study. The Program Director assigned CCMFs to rotations based on a combination of educational opportunities and expected workload demands. Patients admitted to an ICU when a CCMF was on service were assigned to the CCMF plus attending physician group for analysis, while all other patients were assigned to the attending physician group. Information regarding each patient and their hospital stay was obtained from TRACER, the local electronic ICU data warehouse.15 Primary outcomes were ICU and in-hospital mortality, and length of stay (LOS). Secondary outcomes included number of invasive procedures and laboratory investigations per patient and number of ICU readmissions within 72 hours of discharge to the ward. To assess the effect of CCMF stage of training, patient outcomes in the first three blocks of CCMFs training were compared to those in their last three blocks. The local institutional review board approved the study prior to commencement.
At each patient’s ICU admission, the attending physician recorded an admitting diagnosis and level of care (recorded as either “full care provided” or “do-not-resuscitate” [DNR]). Invasive procedures included placement of an endotracheal tube, arterial line, central line (sheath introducer catheter, multilumen infusion catheter, or dialysis catheter), jugular venous bulb, pulmonary artery catheter, or chest tube, or performing a bronchoscopy, thoracentesis or lumbar puncture. Laboratory investigations were obtained from the regional ordering system as “units” of order. For example, a complete blood count or an electrolyte panel was defined as one lab investigation. Readmissions were defined as patients that had an unplanned readmission to an ICU within 72 hours of ICU discharge.
Means for continuous data were compared using the Mann-Whitney U test or independent samples Student t-test where appropriate. Categorical data were compared with the use of the chi-square test. All p values < .05 were considered significant. Statistical analysis used SPSS version 13.0.
Factors that were felt to potentially affect outcome or LOS for either the ICU or the hospital were first examined by univariate analysis, and those with a p ≤ .10 were included in multivariate analyses. A priori, a decision was made to include Acute Physiology and Chronic Health Evaluation II (APACHE II) to ensure that patient complexity was incorporated in the final model. Logistic regression was used to examine the effect on ICU and hospital outcome. Sequential elimination of variables using likelihood ratios method was conducted. Only the “final parsimonious” models following elimination of nonsignificant variables are reported here. Multiple linear regression analysis was used to examine the variables affecting ICU and hospital LOS. The following variables were included in the initial logistic regression model for ICU mortality: CCMF or attending physician as admitting service, age, sex, admitting diagnosis, admission block, admission hospital, APACHE II score, Therapeutic Intervention Scoring System (TISS) score, admission level of care, ICU LOS, number of lab tests and procedures performed, whether the patient was a readmission to the ICU, level of CCMF training and ICU occupancy. The same variables plus hospital LOS were used for the hospital mortality analysis. For the ICU and hospital LOS regression analysis, the same independent variables as the ICU mortality model were included, with the exception of the variable ICU LOS for the ICU LOS model.
Between July 1, 2000 and June 30, 2003, 6,565 patients were admitted to adult ICUs in the CHR. During the study period, seven CCMFs rotated in the ICUs. Their base specialty training before CCMF training were internal medicine and its subspecialties (5), anaesthesia (1), and general surgery (1). There were a total of 22 CCM attending physicians.
CCMFs and attending physicians were on service together for 52 blocks (44.4%) and attending physicians alone for 65 blocks (55.6%). There were 3,341 admissions when attendings and CCMFs were on service together and 3,224 when attending physicians were on alone, resulting in an average of 2.3 and 1.8 admissions per day, respectively (p < .01). Baseline patient characteristics are outlined in Table 1. The patients that CCMFs admitted were younger, more often male and more likely to be admitted with a surgical, trauma, or neurological diagnosis.
For the primary outcome of ICU or hospital mortality, there was no statistically significant difference when both an attending physician and a CCMF were in the ICU as compared to when the attending physician was alone (Table 1). ICU occupancy was 86.0% and 84.0% of capacity when CCMFs and attending physicians or attending physicians alone were on service, respectively (p < .001).
When a CCMF was on service, there were significantly more invasive procedures performed and more laboratory investigations ordered (Table 1). However, there were no differences in the number of patients readmitted within 72 hours of discharge from the ICU.
When comparing fellows during their first three blocks of training to their last three blocks, there was no statistically significant difference in ICU or hospital mortality, or ICU or hospital LOS. However, CCMFs in their first three blocks of training had more patients readmitted within 72 hours of being discharged from the ICU (43 vs. 17; p < .02).
The multivariable models showed that the presence or absence of a CCMF had no effect on ICU or hospital mortality, whereas a higher APACHE II score, an admission DNR order, and admission diagnoses of a neurologic nature, as compared to a medical diagnosis, were associated with an increase in ICU and hospital mortality. A higher number of invasive procedures and a surgical or trauma admission diagnosis were associated with a decrease in ICU and hospital mortality. The number of invasive procedures, APACHE II score, TISS score, and admission to ICU site three accounted for 42% of the variance in a model predicting ICU LOS (Table 2). Readmission to ICU, number of invasive procedures, admission to ICU site two or three, ICU LOS, APACHE II score and TISS score predicted hospital LOS (Table 2).
To our knowledge, this is the first study that has attempted to examine the effect that CCMFs have on patient outcomes. We found that CCMFs do not affect patient mortality rates or LOS in adult ICUs, a result that remained after controlling for any differences in obvious potential confounding factors such as severity of illness, admission diagnosis and admission DNR order.
Two previously published studies demonstrated decreased mortality rates in centers with CCMF training programs.13,14 However, neither of these previous studies was able to discern what aspect of those centers led to the improvement in mortality. Our study contributes to this prior work by specifically examining the effect of CCMFs on patient outcomes in an adult multisystem critical care environment. Our study’s design limits the effect of interinstitutional variability as an alternative explanation for the observed results. The results suggest the mortality benefit found in the previous studies cannot be attributed to the fellows themselves. Rather, the better outcomes seen at teaching hospitals may be due to a combination of factors present within an academic environment such as better access to consultants, research protocols, the latest technology and ancillary resources, and an advanced organizational structure that serves to facilitate care delivery.
Contrary to a number of studies in the literature,13,14,16 we found that CCMFs are not associated with a change in LOS. Instead, multivariable analysis demonstrated that patient-related factors such as admitting diagnosis and severity of illness, and institution-related factors such as ICU LOS and ICU site, play a more important role in predicting mortality and LOS in our patient population. This finding builds on the hypothesis that differences in resource utilization between academic and nonteaching hospitals might be due to characteristics that are inherent within academic training centers; examples in this case might include the trend to admit patients with higher severity of illness or the greater availability and use of technology.13
Our study corroborates the observation that more experienced trainees result in shorter hospital LOS and costs.17 Even within the short two-year period of the ICU training program there was a significant decrease in the number of ICU readmissions when senior CCMFs were on service. As patients unexpectedly readmitted to an ICU have a longer hospital LOS, this result has potential clinical significance.
Our study has a number of limitations. We only used hospitals within one metropolitan center in Canada, therefore caution should be used in generalizing these results to other centers. Previous studies assessed outcomes in only pediatric or trauma ICU patients: two homogeneous patient populations. Our study used a heterogeneous population of adult mixed medical/surgical patients. Although this may limit direct comparisons, the findings of this study remain important in that they help generate new hypotheses regarding root causes for the improved outcomes seen at academic centers.
With increasing calls for greater accountability in medical education, there have been suggestions that the most relevant endpoint to assess the impact of an educational intervention should be clinical outcomes.18,19 At the completion of their training, CCMFs will have been integrated within a medical education environment for nearly a decade or more. Therefore, measuring the effect that these senior trainees have on outcomes could be used as part of a general assessment of the efficacy of an institution’s medical education curriculum. Although the results of our study suggest that CCMFs currently have a neutral impact on patient outcomes, they also suggest there is the opportunity to improve the educational experience within our institution in hopes that future trainees will result in improved patient outcomes.
With the patient safety movement having become a major driver in the realm of patient care delivery, the need to train medical students, residents and fellows to become the next generation of competent physicians must be balanced with the potential risk of increased medical errors as these trainees gain experience and learn. The results of this study should be reassuring to medical educators, administrators and patients in that in an environment that provides appropriate support and supervision, experienced trainees do not negatively affect patient outcomes.
1 Yuan Z, Cooper GS, Einstadter D, Cebul RD, Rimm AA. The association between hospital type and mortality and length of stay: a study of 16.9 million hospitalized Medicare beneficiaries. Med Care. 2000;38:231–45.
2 Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severity-adjusted mortality and length of stay in teaching and nonteaching hospitals. Results of a regional study. JAMA. 1997;278:485–90.
3 Kupersmith J. Quality of care in teaching hospitals: a literature review. Acad Med. 2005;80:458–66.
4 Offner PJ, Hawkes A, Madayag R, Seale F, Maines C. General surgery residents improve efficiency but not outcome of trauma care. J Trauma. 2003;55:14–9.
5 Jackson JL, Kroenke K, Pangaro L. A comparison of outcomes for walk-in clinic patients who see interns and those who see staff physicians. Acad Med. 1999;74:718–20.
6 Moore JD Jr. Teaching hospitals have better outcomes. Mod Healthc. 1999;29:34–6.
7 Griffith CH 3rd, Rich EC, Hillson SD, Wilson JF. Internal medicine residency training and outcomes. J Gen Intern Med. 1997;12:390–6.
8 Stranjalis G. Ruptured cerebral aneurysm: influence of specialist and trainee-performed operations on outcome. Acta Neurochir. 1996;138:1067–9.
9 Salazar A, Corbella X, Onaga H, et al. Impact of a resident strike on emergency department quality indicators at an urban teaching hospital. Acad Emerg Med. 2001;8:804–8.
10 Fallon WF Jr., Wears RL, Tepas JJ 3rd. Resident supervision in the operating room: does this impact on outcome? J Trauma. 1993;35:556–60.
11 Simmer TL, Nerenz DR, Rutt WM, Newcomb CS, Benfer DW. A randomized, controlled trial of an attending staff service in general internal medicine. Med Care. 1991;29:JS31–40.
12 Pollack MM, Cuerdon TT, Patel KM, et al. Impact of quality-of-care factors on pediatric intensive care unit mortality. JAMA. 1994;272:941–6.
13 Pollack MM, Patel KM, Ruttimann E. Pediatric critical care training programs have a positive effect on pediatric intensive care mortality. Crit Care Med. 1997;25:1637–42.
14 Arbabi S, Jurkovich GJ, Rivara FP, et al. Patient outcomes in academic medical centers: influence of fellowship programs and in-house on-call attending surgeon. Arch Surg. 2003;138:47–51.
15 Doig CJ, Zygun DA, Fick GH, et al. Study of clinical course of organ dysfunction in intensive care. Crit Care Med. 2004;32:384–90.
16 Higgins TL, McGee WT, Steingrub JS, et al. Early indicators of prolonged intensive care unit stay: impact of illness severity, physician staffing and pre-intensive care unit length of stay. Crit Care Med. 2003;31:45–51.
17 Rich EC, Gifford G, Luxenberg M, Dowd B. The relationship of house staff experience to the cost and quality of inpatient care. JAMA. 1990;263:953–7.
18 Chen FM, Bauchner H, Burstin H. A call for outcomes research in medical education. Acad Med. 2004;79:955–60.
19 Lim JK, Golub RM. Graduate medical education research in the 21st Century and JAMA On Call. JAMA. 2004;292:2913–5.