Dine, C Jessica; Bellini, Lisa M.; Iwashyna, Theodore J.
Although graduate medical education (GME) is subsidized by several sources, there remains substantial concern about the added costs of care provided by physicians-in-training. Increased costs result not only from the tendency of less experienced physicians to order more testing than more seasoned physicians1 but also because residents may order labs more variably and less in response to clear clinical indications. This variation can itself induce costs by requiring extra capacity to accommodate those swings in demand. In an era of increasing concerns about the fiscal viability of academic medical centers, reducing variation in laboratory utilization may lessen the costs of GME.
Understanding variations of laboratory utilization, however, may have other impacts on GME. Physicians-in-training receive little information regarding their own practice pattern. Improved knowledge about their own practice patterns and hospital costs might reduce resource utilization and better prepare physicians-in-training for today’s health care market. In fact, the Accreditation Council for Graduate Medical Education (ACGME) included practice-based learning and improvement (PBLI) and systems-based practice (SBP) in the six core competencies at the heart of resident education2—competencies that directly reflect resident self-conscious awareness of their own patterns of testing and other resource utilization.
Although many medical educators suspect that trainee variation may induce extra variation in practice, the specific patterning of this has not been well studied. Does significant variation in resource utilization between admissions that cannot be explained by clinical characteristics of patients exist? If so, what proportion of this variation can be explained by the intern or resident as opposed to the practice patterns of the attending of record? If such variation exists, it is likely to be of little benefit to the patient and can be a target for future educational interventions to not only increase the knowledge of physicians-in-training about PBLI and SBP but also potentially reduce the cost of GME.
The purpose of the study was to test the following hypothesis: There exists variation in laboratory utilization between patient-days on three subspecialty services that is attributable to the identity of particular care team members independent of patient characteristics.
This study was conducted at the Hospital of the University of Pennsylvania (HUP), a tertiary-care academic medical center, after approval by the institutional review board (IRB). Data on laboratory utilization were collected between June of 2007 and January of 2008. Laboratory data of patients on the Oncology Service and in the Medical Intensive Care Unit (MICU) were collected.
The Oncology Service is divided into teams caring for patients with hematologic (Liquid Oncology) and solid (Solid Oncology) malignancies. The teams on the Oncology Service are composed of interns under the supervision of a resident and an oncology attending. The MICU at HUP is a 24-bed unit. We examined medical teams of the same structure as on the Oncology Service—interns under the supervision of a resident who, in turn, is supervised by an attending physician.
Provider identification and schedule
Once the patient is admitted, the intern assigns the patient to him- or herself electronically. Because the end-of-shift sign-out system is predicated on this electronic assignment, compliance is high and self-policed. This information is then matched with the electronic scheduling system used by the medicine department at HUP to identify the senior resident and attending physician involved in the care of the individual patient on that day. Interns and residents typically rotate on each service for one month and are on call every third night in the MICU and every fourth night on the Oncology Service. The attending physician coverage is typically for 14 days. MICU attending physicians are on call each night, whereas oncology attending physicians cover calls every other night. During the eight months of data collection, interns, residents, and attending physicians rotate frequently. Interns and residents rotate every four weeks but are offset by one week. Furthermore, attendings rotate every two weeks. These frequent and offset rotations allow for distinct team configuration and, thus, statistical identification of variance of laboratory utilization by provider type.
A patient-day was defined as a period from noon to noon. This accounts for the fact that most of the morning laboratory orders were actually placed the evening prior to that day. Labs ordered or collected in the Emergency Department, in the outpatient setting, or by another department before transfer to medicine were excluded from the analysis. Labs were only assigned to the MICU team if they were ordered during the MICU stay. Likewise, MICU laboratory tests were excluded for the Oncology Service. The patient-day was assigned to the ICU if the patient spent the majority of the day in the ICU.
Laboratory tests were aggregated into composite laboratory tests if they were ordered at the same time. For example, white blood cell count, hemoglobin, and platelet count were aggregated into a single laboratory test of a complete blood count, even if they were ordered separately, as long as they were drawn together. Point-of-care fingersticks for blood glucose levels were excluded because these are routinely done by nursing without physician scheduling. The laboratory costs were calculated using the marginal variable supply costs obtained from the hospital cost-accounting system.
Ordinary least-squares regression was used for the analysis to explain the variance in laboratory utilization by provider type.3 The overall explained variation was defined as the R2 for a model containing the appropriate provider type. Patients were cared for by a primary intern under the supervision of a resident and an attending. The contribution of each provider type to the variance in laboratory utilization was calculated by dividing the difference of the R2 of the models including and excluding the provider type by the overall explained variation. Bootstrapping by taking 1,000 repeated samples with replacement from the observed population of patient-days at the patient level was performed for confidence intervals.4,5 Sequential F testing was performed as a complementary approach to detect whether adding one provider type to the model already containing the other two would increase the explanatory power of the model. Data were analyzed by using STATA 10.0 (StataCorp, College Station, Texas).
During data collection, 23 interns, 15 residents, and 17 attending physicians served on the Liquid Oncology Service. There were 123 patients with 188 total hospitalizations comprising 1,598 patient-days. Characteristics of patients, resource utilization, patient outcomes, and providers are shown in Table 1.
Team composition alone on the liquid oncology service explained 11.1% of variation between non-ICU patient-days in lab utilization. The attending explained 14.0% (95% CI: 9.8%, 27.3%) of that team-effect variation, with 20.8% (95% CI: 12.5%, 30.5%) explained by the resident and 37.3% (95% CI: 26.5%, 48.2%) by the intern. The remainder could not be attributed to any member of the team.
Thirty-three interns, 18 residents, and 23 attending physicians rotated on the Solid Oncology Service. Five of these interns and one of the residents had also rotated on the Liquid Oncology service during data collection. There were 431 hospitalizations of 302 patients with a total of 2,322 patient-days. On this service, 7.4% of the overall variation in laboratory use could be explained by providers as opposed to patients. Attendings accounted for 21.9% (95% CI: 14.5%, 35.2%) of the explained variation in daily laboratory ordering, residents 15.0% (95% CI: 8.2%, 25.0%), and interns 45.0% (95% CI: 32.2%, 55.1%). The remaining 18.1% could not be uniquely attributed to any team member.
For both oncology services, a complementary approach was used to detect whether adding one provider type to the model already containing the other two would increase the explanatory power of the model, as measured by sequential F testing. For any combination, the explanatory power was increased each time until both attending physicians and interns were in the model. However, if both of these provider types were already in the model, adding residents no longer increased the explanatory power.
Medical intensive care unit
Data were collected on 270 patients who were covered by 22 interns, 22 residents, and 14 attendings during 286 hospitalizations for a total of 1,247 ICU patient-days. Thirteen of these interns had also rotated on the Liquid and Solid Oncology services. There was no overlap of residents or attending physicians. Patient characteristics and data on resource utilization including average number and costs of laboratory tests ordered in the MICU are shown in Table 1. For patients cared for by an intern under the supervision of a senior resident and an attending, the team explained 12.5% of the day-to-day variation in lab utilization. The attending explained 11.8% (95% CI: 6.1%, 25.6%) of the variation, and the resident explained 26.3% (95% CI: 15.8%, 39.4%). The intern accounted for 25.6% (95% CI: 13.0%, 39.1%), and 36.3% could not be uniquely attributed to any team member. Sequential F tests confirmed that each provider type added explanatory value to a model already containing the other two.
Variation in laboratory utilization between patient-days on three subspecialty services exists. The identities of the caring physicians explain a proportion of this variation (11.1%, 7.4%, and 12.5% for the Liquid Oncology, Solid Oncology, and MICU services, respectively). Although this is a modest proportion of the overall variation observed, it has no clear benefits for patients and may be amenable to intervention. It represents substantial variable costs that are directly removed from the financial margins of medical centers. Housestaff are responsible for a portion of the variation attributable to physicians. This may represent an opportunity for quality improvement and cost containment at academic health centers via strategies to reduce such variation.
Some investigators have suggested that physician personal attributes, such as age and gender, affect their ordering profiles.6 However, physician understanding and knowledge of current guidelines have also been shown to contribute to the variation.7 By understanding the etiologies of such variation, it may be possible to reduce it in laboratory ordering through improvements in medical training and education. In particular, educational interventions may increase the knowledge of physicians-in-training about their resource utilization and provide an opportunity for formal training on PBLI and SBP as mandated by the ACGME. This could also improve the efficiency of their practice.
The variation in laboratory utilization may also present a possibility to improve the teaching and supervision of medical residents. To minimize variation in resource utilization, any intervention will have to be aimed at the health care team with particular attention to the predominant driver in resource utilization. Interns may have the predominant impact on variations of laboratory utilization in our case. This impact seems to decrease with increases in the level of attending supervision. For these three services at HUP, the level of supervision increases in the following order: Solid Oncology < Liquid Oncology < MICU. The interns on these services explained 45%, 37%, and 26%, respectively, of the variation in laboratory ordering. Future studies will have to be done to determine whether these means in proportion of team variation explained by interns were significantly different across these three services.
Understanding variations in resource utilization may also lead to cost containment while improving quality of care provided. For example, variations in laboratory ordering may result in unnecessary work-ups of false-positive tests.8,9 The investigators of one study published more than 30 years ago demonstrated that a large variation in laboratory utilization existed among similarly trained physicians caring for similar patients.10 Later, they demonstrated that there was a strong correlation between an intern’s overall cost rank among his or her peers and the number of unnecessary tests ordered.11
This is a single-center study. The variation between patient-days may differ at other institutions. A direct link between variation in laboratory utilization and the quality of care provided has not been made. Future work will investigate this further. The amount of physician contribution to the overall variation in laboratory ordering is small. However, it may present a target for future interventions, may be a visible signal of many similar practice variations throughout processes of care, and may still provide a mechanism for potentially large cost reduction.
Variation exists in laboratory utilization between patient-days attributable to physicians on three subspecialty services. Housestaff explain a large proportion of this variation. This may represent an important opportunity for quality improvement and cost containment at academic health centers via strategies to reduce such variation while simultaneously addressing the ACGME core competencies of systems-based practice and practice-based learning.
1 Khaliq AA, Huang CY, Ganti AK, et al. Comparison of resource utilization and clinical outcomes between teaching and nonteaching medical services. J Hosp Med. 2007;2:150–157.
2 Goroll AH, Sirio C, Duffy FD, et al. A new model for accreditation of residency programs in internal medicine. Ann Intern Med. 2004;140:902–909.
3 Cohen J, Cohen P, West SG, et al. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 2003.
4 Good PI. Resampling Methods. Boston, Mass: Birkhauser; 2006.
5 Mooney CZ, Duval R. Bootstrapping: A Nonparametric Approach to Statistical Inference. Newbury Park, Calif: Sage Publications; 1993.
6 Kristiansen IS, Hjortdahl P. The general practitioner and laboratory utilization: Why does it vary? Fam Pract. 1992;9:22–27.
7 Solomon DH, Hashimoto H, Daltroy L, et al. Techniques to improve physicians’ use of diagnostic tests: A new conceptual framework. JAMA. 1998;280:2020–2027.
8 Boohaker EA, Ward RE, Uman JE, et al. Patient notification and follow-up of abnormal test results. A physician survey. Arch Intern Med. 1996;156:327–331.
9 DeKay ML, Asch DA. Is the defensive use of diagnostic tests good for patients, or bad? Med Decis Making. 1998;18:19–28.
10 Schroeder SA, Kenders K, Cooper JK, et al. Use of laboratory tests and pharmaceuticals. Variation among physicians and effect of cost audit on subsequent use. JAMA. 1973;225:969–973.
11 Schroeder SA, Schliftman A, Piemme TE. Variation among physicians in use of laboratory tests: Relation to quality of care. Med Care. 1974;12:709–713.