O'Connor, Alec B. MD, MPH; Lang, Valerie J. MD; Lurie, Stephen J. MD, PhD; Lambert, David R. MD; Rudmann, Andrew MD; Robbins, Brett MD; Bordley, Donald R. MD
Dr. O'Connor is assistant professor of medicine, Department of Medicine, and associate program director, Medicine and Medicine/Pediatrics Residencies, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Lang is assistant professor of medicine, Department of Medicine, and director, Internal Medicine Clerkship and Subinternship, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Lurie is assistant professor of family medicine, Department of Family Medicine, and director of assessment, Office of Curriculum and Assessment, University of Rochester School of Medicine and Dentistry.
Dr. Lambert is associate professor of medicine, Department of Medicine, and senior associate dean for medical student education, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Rudmann is associate professor of medicine, Department of Medicine, and chief, Hospital Medicine Division, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Robbins is associate professor of medicine and pediatrics, Departments of Medicine and Pediatrics, and program director, Medicine/Pediatrics Residency, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Dr. Bordley is professor of medicine, Department of Medicine, program director, Medicine Residency, and associate chair for education, University of Rochester School of Medicine and Dentistry, Rochester, New York.
Please see the end of this article for information about the authors.
Correspondence should be addressed to Dr. O'Connor, Box MED/HMD, Strong Memorial Hospital, 601 Elmwood Avenue, Rochester, NY 14642; telephone: (585) 275-4912; fax: (585) 276-2144; e-mail: (firstname.lastname@example.org).
In the recent past, residents cared for virtually all medical inpatients in many teaching hospitals,1,2 including our own. However, the volume and complexity of inpatients in teaching hospitals have increased substantially during the past decade.1,3–6 At the same time, there has been increased emphasis on outpatient resident training and limiting resident duty hours. In response to these factors, many teaching hospitals have created nonteaching services (NTSs), some of which employ midlevel providers (MLPs; i.e., nurse practitioners or physician assistants) to function in the traditional resident's role.1,2,4,5,7–15 In the two hospitals affiliated with our residency programs, independent NTSs were created in 1995 and 1999. Both NTSs have grown rapidly and now care for the majority of medical inpatients in each hospital.
Residents and MLPs differ in their training, abilities, and goals, so they may not provide completely interchangeable forms of coverage.11 Randomly dividing patients between teaching services and NTSs may not be possible in many hospitals for a variety of reasons, including MLPs' comfort managing patients with certain diagnoses or high acuity and the timing of NTS staffing. It is likely that these patient-selection pressures could cause teaching services to consist of patients who are sicker and carry different diagnoses than are characteristic of the pool of patients admitted to the hospital.
At its core, graduate medical education is experiential.1,3,5,16,17 Exposure to illnesses is critical to recognizing clinical patterns and developing clinical reasoning skills.18 Although the optimal educational mix of inpatients has not been determined, a logical goal is for residents' patients to be representative of the population of patients for whom they will care when they become attendings. If the difference between the inpatients that medical residents care for and the overall population of all medical inpatients is substantial, then the presence of NTSs may contribute to the gap between residency training and internal medicine practice.4
We hypothesized that patients cared for by our medical residents have higher acuity and different principal diagnoses than patients cared for by our mature NTS. We sought to quantify the differences between the patients on the teaching services and NTSs.
We included only medical patients admitted to general medical floors in either of two teaching hospitals affiliated with the University of Rochester School of Medicine and Dentistry in Rochester, New York: Strong Memorial Hospital, a 750-bed tertiary care referral center; and Highland Hospital, a 268-bed community hospital. The data span the period from January 1, 2005 to June 30, 2005. The combined data were for 6,907 patients, of whom 1,976 (29%) were covered by medical residents and 4,931 (71%) were covered by NTSs. At the time of the study there were 72 categorical medicine residents and 32 medicine–pediatrics residents, all of whom rotated onto inpatient medicine services in both hospitals.
All data were obtained retrospectively by linking two existing databases. The first is a computerized sign-out program that is used by all medical residents and MLPs at both hospitals. Coverage assignment in the sign-out database is based on the coverage at the time of discharge or death. Patients remained on the initially assigned coverage team except in two situations. First, patients who were deemed by the covering MLPs to be too unstable or time consuming were permanently transferred to the resident service. Although transfer of coverage is not recorded in the database, our experience tells us that such transfers are infrequent. Second, patients who were transferred to the intensive care unit (ICU) were reassigned to new coverage (either resident or MLP), independent of prior coverage, upon transfer out of the ICU if the duration of ICU stay exceeded five days.
The coverage assignments, as defined by the sign-out databases, were linked to the hospitals' billing databases using patient admission identification numbers. All reported data except the coverage assignment were derived from the hospitals' billing databases.
At Strong, the NTS originated 10 years before data collection and consisted of 21 full-time nurse practitioners or physician assistants. Coverage at Highland consists of several different parallel options, including medicine residents, family medicine residents, geriatrics fellows, and physician assistants. To focus on the patients seen by the medicine residents relative to all of the patients admitted for medical diagnoses, all patients not cared for by the medicine residents are considered to be on the NTS in this analysis (i.e., in our primary analysis the NTS at Highland consists of a mixture of patients covered by physician assistants, family medicine residents, and geriatrics fellows, though more than three fourths of these patients were covered by the physician assistants). The physician assistant service at Highland was six years old at the time of the study, consisted of six physician assistants, and is completely independent of the NTS at Strong. In both hospitals, members of the NTS are employed by the hospital, and the medicine residency does not have administrative oversight. Both resident services and NTSs provide 24-hour-per-day coverage in both hospitals.
At Strong, patients were assigned to either the NTS or resident service by a nurse coordinator, sometimes in consultation with a senior resident. At Highland, coverage assignments of patients admitted overnight by the medical night float team were made by the medical chief resident, whereas daytime admissions were called sequentially to the resident service until a cap was reached. Coverage assignments at both hospitals were based on the availability of coverage and, in some cases, patient acuity, because some MLPs in both hospitals are uncomfortable caring for the sickest patients. Occasionally, patients with unusual diagnoses considered to be especially good “learning cases” were preferentially placed on the resident service. Coverage assignments are independent of the attending physician and patient insurance.
ICD-9 codes were collapsed to whole numbers in order to group similar diagnoses (e.g., different variations of uncontrolled diabetes were all considered uncontrolled diabetes). (ICD-9 stands for International Statistical Classification of Diseases and Health Problems, Ninth Revision, a universally used coding system for diagnoses and symptoms.) Charlson Comorbidity Index (CCI) scores were calculated from the hospital database ICD-9 diagnoses using a modified version of a publicly available SAS macro.19,20
We used two-sided Wilcoxon rank sum tests and Fisher exact tests for the hypothesis testing of medians and proportions, respectively. The study was approved by the University of Rochester Research Subjects Review Board. Statistical analyses were performed using SAS version 9.1 (Cary, North Carolina).
Differences in summary measures
As stated earlier, when data from the two hospitals were combined, there were a total of 6,907 patients, of whom 1,976 (29%) were covered by medicine residents. Patients on the resident service were younger and more likely to be male, African American, and Medicaid insured, but the proportion of hospitalist-attended patients was similar (Table 1).
Compared with the NTS patients, patients on the resident service had higher CCI scores, a greater number of comorbidities, longer lengths of stay, higher operating costs, and were more likely to receive intensive care and to die in the hospital (Table 1).
Patient demographics differed between the hospitals: compared with patients at Strong, patients at Highland were older, more likely to be Medicare insured, and less likely to be attended by a hospitalist physician. Nevertheless, between-service differences in patient complexity and disposition were similar to those of the combined hospital population.
At Highland there were 432 patients covered by family medicine residents and 37 by geriatrics fellows (14.6% and 1.3% of the total Highland population, respectively); removal of these patients from the analysis produced a small increase in the differences between the teaching services and NTSs, but it did not substantially change the results (data not shown).
For the entire cohort, between-service differences persisted when excluding patients with an ICU stay and when analyzing patients with each of the three most common diagnoses (data not shown).
Differences in principal diagnoses
The most common principal diagnoses cared for by either service are shown in Table 2. Compared with the NTS providers, residents were more likely to care for patients with principal diagnoses of acute renal failure, respiratory failure, septicemia, and HIV. Residents were less likely to care for patients with principal diagnoses of pneumonia, chest pain, cellulitis, intestinal obstruction, diverticular disease, alcohol withdrawal, abdominal pain, and sickle cell crisis.
The overall frequencies of diagnoses were generally similar in the two hospitals, except that chest pain, aspiration pneumonitis, and cerebrovascular accident were more common diagnoses at Highland. Substantial and statistically significant preferential distribution of patients onto the resident service occurred at both hospitals for patients with acute renal failure, respiratory failure, and septicemia, whereas patients with chest pain were preferentially assigned to the NTSs at both hospitals (Table 2).
We found that compared with patients on the NTSs, patients admitted to the resident service were demographically different, had higher comorbidity burdens, and were more likely to have high-acuity diagnoses. Because residents will be, upon graduation, responsible for caring for the entire pool of medical inpatients, our findings demonstrate that an NTS can be associated with a training–practice gap.4
Our findings contrast with those of a recent study comparing outcomes between teaching services and NTSs in a teaching hospital in Oklahoma.21 The authors found that acuity, diagnoses, and outcomes (including costs, length of stay, readmission rates, and in-hospital mortality) were similar between teaching service and NTS patients. In contrast to our NTS, admissions to their NTS were randomly assigned, NTS patients were apparently covered by hospitalists without MLPs, and their NTS was comparatively small (25% of all admissions). We believe that the random allocation of patients between their teaching services and NTSs accounts for the similarities in their teaching and nonteaching patient populations. We suspect that the differences in outcomes between patients on our teaching services and NTSs are largely attributable to baseline differences in patient demographics and acuity, and that adjusting for these factors would result in more similar patient outcomes.
Intentional manipulation of residents' case mix can potentially benefit residents' education. The implementation of an NTS can reduce the volume of patients that residents care for, including those patients thought to have less educational potential,9,12,15 such as patients with overrepresented diagnoses or admitted for procedures or nursing home placement, which may improve educational outcomes.1,6 However, our findings indicate that NTSs can unintentionally result in both higher acuity and altered case mix among residents' patients. Reduced exposure to “bread and butter” diagnoses, such as chest pain and cellulitis, might worsen residents' educational experience if they are not exposed to sufficient volumes of patients with these diagnoses.
It is possible that caring for more complex, higher-acuity patients is beneficial to residents, because these may be the most challenging patients to care for. However, it is also possible that our findings could be associated with negative or unexpected outcomes. Residents' self-rated preparedness and the development of clinical reasoning skills depend on the amount of exposure to specific illnesses.18,22 It is therefore possible that relative underexposure to certain illnesses could contribute to residents' being unprepared by the time they graduate. Relative overexposure to high-acuity patients might affect residents' clinical probability estimates in, for example, future chest pain patients. Also, caring for consistently high-acuity patients could potentially contribute to the development of burnout among residents. Finally, it is possible that the acuity and case mix changes associated with a large NTS could affect students' and residents' career choices.
We have redesigned the inpatient services at both hospitals in an effort to address our findings. One of the primary goals of our redesign is to achieve a more random allocation of patients between services. To assist our MLPs in caring for sicker patients, we are designating a senior resident to assist them in particularly challenging situations; we hope that this will allow high-acuity patients to remain on the NTS, which will allow the teaching service patient population to better represent the entire population of medical inpatients.
This study has several limitations. Because NTSs are likely to evolve in a way that is highly dependent on the needs of a given hospital, there are likely to be many different types of NTSs. In short, the interface between a residency program and an NTS is likely to be highly institution-specific. However, we found consistent differences between the teaching services and NTSs in our two very different hospitals with independent NTSs. The pressures driving the development of NTSs in our hospitals are similar to those in hospitals throughout the United States.1,5,6 Furthermore, the factors affecting the distribution of patients between services in our hospitals have been described in other hospitals.9,11,13
Our data are derived from hospital billing records, which might contain errors in coding. We were unable to quantify the number of patients transferred to the teaching service during their hospitalization due to the acuity or overall complexity of their conditions. However, we believe, on the basis of experience, that these transfers are relatively rare, and we believe that if we could have quantified this frequency, that would not have substantially changed our results or conclusions.
The nature of the experience on inpatient teaching services has changed since the introduction of NTSs in teaching hospitals. The growth of NTSs has not been accompanied by assessment of the impact of NTSs on education. Research defining the impact of NTSs on residents' and students' education is needed.
The authors are indebted to members of the Decision Support Offices at Strong Memorial Hospital and Highland Hospital for providing the relevant hospital database information.
An abstract summarizing this report was presented at the Association of Program Directors in Internal Medicine's spring conference, April 17, 2007, in San Diego, California.
1 Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC. Redesigning residency education in internal medicine: A position paper from the Association of Program Directors in Internal Medicine. Ann Intern Med. 2006;144:920–926.
2 Simmer TL, Nerenz DR, Rutt WM, Newcomb CS, Benfer DW. A randomized, controlled trial of an attending staff service in general internal medicine. Med Care. 1991;29:JS31–JS40.
3 Smith LG, Humphrey H, Bordley DR. The future of residents' education in internal medicine. Am J Med. 2004;116:648–650.
4 Arora V, Guardiano S, Donaldson D, Storch I, Hemstreet P. Closing the gap between internal medicine training and practice: Recommendations from recent graduates. Am J Med. 2005;118:680–685.
5 Lundmerer KM, Johns MME. Reforming graduate medical education. JAMA. 2005;294:1083–1087.
6 Weinberger SE, Smith LG, Collier VU. Redesigning training for internal medicine. Ann Intern Med. 2006;144:927–932.
7 Spisso J, O'Callaghan C, McKennan M, Holcroft JW. Improved quality of care and reduction of housestaff workload using trauma nurse practitioners. J Trauma. 1990;30:660–665.
8 Knickman JR, Lipkin M, Finkler SA, Thompson WG, Kiel J. The potential for using non-physicians to compensate for the reduced availability of residents. Acad Med. 1992;67:429–438.
9 Goksel D, Harrison CJ, Morrison RE, Miller ST. Description of a nurse practitioner inpatient service in a public teaching hospital. J Gen Intern Med. 1993;8:29–30.
10 Genet CA, Brennan PF, Ibbotson-Wolff S, et al. Nurse practitioners in a teaching hospital. Nurse Pract. 1995;20:47–52.
11 Pioro MH, Landefeld CS, Brennan PF, et al. Outcomes-based trial of an inpatient nurse practitioner service for general medical patients. J Eval Clin Pract. 2001;7:21–33.
12 Wilson SD. Employing hospitalists to improve residents' inpatient learning. Acad Med. 2001;76:556.
13 Kulaga ME, Charney P, O'Mahony SP, et al. The positive impact of initiation of hospitalist clinician educators. J Gen Intern Med. 2004;19:293–301.
14 Nishimura RA, Linderbaum JA, Naessens JM, Spurrier B, Koch MB, Gaines KA. A nonresident cardiovascular inpatient service improves residents' experiences in an academic medical center: A new model to meet the challenges of the new millennium. Acad Med. 2004;79:426–431.
15 Myers JS, Bellini LM, Rohrback J, Shofer FS, Hollander JE. Improving resource utilization in a teaching hospital: Development of a nonteaching service for chest pain admissions. Acad Med. 2006;81:432–435.
16 Papadakis MA, Kagawa MK. Categorical medicine residents' experiential curriculum. Am J Med. 1995;98:7–12.
17 Ende J, Davidoff F. What is curriculum? Ann Intern Med. 1992;116:1055–1057.
18 Bowen JL. Medical education: Educational strategies to promote clinical diagnostic reasoning. N Engl J Med. 2006;355:2217–2225.
19 Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619.
21 Khaliq AA, Huang CY, Ganti AK, Invie K, Smego RA. Comparison of resource utilization and clinical outcomes between teaching and nonteaching medical services. J Hosp Med. 2007;2:150–157.
22 Wiest FC, Ferris TG, Gokhale M, Campbell EG, Weissman JS, Blumenthal D. Preparedness of internal medicine and family practice residents for treating common conditions. JAMA. 2002;288:2609–2614.