Share this article on:

A Pilot Study Assessing the Influences of Charge Data and Group Process on Diagnostic Test Ordering by Residents

Rudy, David W. MD; Ramsbottom-Lucier, Mary MD, MPH; Griffith, Charles H. III MD, MPH; Georgesen, John C. PhD; Wilson, John F. PhD

Educating Physicians: Research Reports

Purpose Providing charge data to resident physicians has been shown to reduce the amounts spent on diagnostic testing. This pilot study sought to determine the influences of charge data and group decision making on diagnostic test ordering by internal medicine residents.

Method In an interactive workshop, 23 internal medicine residents received a hypothetical case. They completed an 18-item questionnaire estimating charges for diagnostic tests and then “ordered” tests. The residents were then randomly divided into groups that either received charge data, received charge data after ordering tests, or received no charge data. The groups ordered tests by consensus. Tests were weighted for appropriateness (+1 to +6) and inappropriateness (−1 to −6). Analyses compared individual and group decisions and effect of availability of charge data.

Results Residents with access to charge data spent less on tests, but also had lower appropriateness scores. The appropriateness of the diagnostic workup was better by groups than by individuals, but cost more.

Conclusion Cost-containment interventions targeted towards doctors in training need to address the effect on quality of care and the influence of the group process in clinical decision making. Group diagnostic decisions may be more costly, but more appropriate.

Dr. Rudy is assistant professor, Dr. Ramsbottom-Lucier is assistant professor, and Dr. Griffith is associate professor in the Department of Internal Medicine; Dr. Georgesen is research associate and Dr. Wilson is professor in the Department of Behavioral Medicine. All are at the University of Kentucky College of Medicine, Lexington.

Correspondence and requests for reprints should be addressed to Dr. Rudy, General Internal Medicine, K512 Kentucky Clinic, Lexington, KY 40536-0284; e-mail: 〈Dwrudy0@pop.uky.edy〉.

Presented in part at the annual meeting of the Southern Section of the Society of General Internal Medicine, New Orleans, LA, February 1999, and the Society of General Internal Medicine national meeting, San Francisco, CA, April 1999.

The authors acknowledge Dr. Robert Lightfoot, Dr. Richard Glassock, Dr. Peter Sawaya, and Dr. Mary Duke for their assistance in the development and implementation of the workshop used in this investigation.

Academic medical centers are under scrutiny by payers for providing more costly care than do community hospitals. One reason for the expense may be that groups of doctors caring for patients in a residency program may spend more on diagnostic testing than do individual practitioners.1,2

While a number of interventions, such as the availability of charge data at the time of test ordering,3,4 have been successful in reducing the cost of diagnostic testing by residents, little is known about their influence on the quality of care.5 How does the practice of making decisions by groups of residents influence the cost of diagnostic evaluations and the quality of the care provided?

The purpose of our pilot study was to evaluate the influences of charge data and group decision making on the cost and quality of a diagnostic evaluation performed on a hypothetical case by internal medicine residents. We hypothesized that the availability of charge data would reduce the costs at the expense of the quality of the workup and that the group process in decision making would improve quality but increase cost.

Back to Top | Article Outline


In October 1997, 23 residents (PGY-1 through PGY-4) from our internal medicine and internal medicine-pediatrics programs participated in an interactive workshop on cost-effectiveness in which the residents developed a diagnostic plan for a hypothetical case. The case, modified from a patient management problem in the Medical Knowledge Self Assessment Program (MKSAP) VIII,6 was that of a 17-year-old male with neurofibromatosis and severe hypertension suggestive of secondary causes, such as pheochromocytoma, and the correct diagnosis, renal artery stenosis.

At the beginning of the workshop, the residents completed an 18-item questionnaire that asked them to estimate the charges for various diagnostic tests. After the case was presented, the residents were asked to select diagnostic tests from four sequential menus of options. They first selected their individual evaluations for each list by marking the tests they wished to order on an order sheet. After these were collected, the residents were randomly divided into six groups of three to four and asked to come to a group consensus about which tests to order. Once the group selected a test, the results were revealed on a poster display. This process was repeated for the three remaining lists of tests.

All groups were asked to develop a cost-effective diagnostic plan for the case. To evaluate the influence of the availability of charge data on residents' ordering of diagnostic tests, each group was randomly assigned to one of three strategies: (1) no charge data available, (2) charge data (obtained from hospital databases) revealed after a test was ordered, and (3) charge data available for all tests prior to ordering. Because we found no significant difference between the responses of groups 2 and 3, these groups were combined for analysis.

In addition to charges, each diagnostic test was assigned an appropriateness score, as in the original MKSAP format.6 A panel of five faculty experts updated these scores to meet current practice standards. A test was determined to be appropriate if it was necessary for the evaluation and inappropriate if it was noncontributory. Appropriate tests were weighted based on their importance in making a diagnosis and assigned values of +1 to +6, and inappropriate tests were assigned values of −1 to −6 based on inconvenience and risks (e.g., renal arteriogram +6, kidney biopsy −6).

The effects of charge data on costs and appropriateness scores were examined using t-tests for independent groups, comparing the 15 residents who received charge data with the eight residents who did not receive charge data. The effects of group process were examined using the group as the unit of analysis, with paired t-tests comparing the mean cost and appropriateness scores of individuals in a group prior to group consensus to the scores obtained after group consensus.

Back to Top | Article Outline


The resident groups did not differ by postgraduate year (2.2 versus 2.3, p = .88) or in their accuracy of estimating charges for diagnostic tests (p = .85). As can be seen in Table 1, the residents who had access to charge data spent considerably less on testing than did the residents who did not have access to the data ($1,297 versus $2,205; p = .03), but the residents with charge data had a lower overall appropriateness score (12.3 versus 18.8; p = .01). The influence of group decision making is shown in Table 2. Groups, as opposed to individuals, spent more on the diagnostic evaluation ($2,505 versus $1,626; p = .001). Groups also ordered more inappropriate tests than did individuals (−10 versus −5.5; p = .002), but critically, they also ordered more of the appropriate tests (25 versus 19.8; p = .01).

Table 1

Table 1

Table 2

Table 2

Back to Top | Article Outline


In summary, the availability of charge data decreased the costs of the diagnostic tests ordered by individual residents, but it was also associated with decreases in the total appropriateness scores of the diagnostic tests ordered. Group decision making was more costly than was the decision making by individuals, but the groups ordered more appropriate tests, which improved the diagnostic accuracy of the evaluation. However, the groups also ordered more inappropriate tests, which contributed to the higher costs of the groups' evaluations.

There are several limitations to our study. The case was hypothetical and did not include the psychosocial and financial considerations of an actual patient encounter. However, this may have helped to isolate diagnostic reasoning based upon the differential diagnosis, the predictive value, and the costs of the tests from any confounding issue. The case was also complex, but we felt that it reflected the complexity of diagnostic dilemmas often referred to tertiary care centers. Even though the case was rare, the main diagnostic strategy, evaluation of secondary causes of hypertension, is not an uncommon problem. Although sample size was small, the effect size was large and the size of the resident groups reflected the size of a typical ward team. While we assessed residents' decision making only, the attending physician and his or her interaction with the residents can also influence test ordering.7

Teaching hospitals may provide a higher quality of patient care at a higher cost than do non-teaching hospitals.8 Could the increased accuracy of diagnosis by groups of doctors be partially responsible for this finding? Studies indicate that groups of residents most likely improve the diagnostic evaluation by pooling their knowledge bases and expanding the differential diagnosis.9 This may result in a more costly and, also, a more appropriate evaluation. In our pilot study, providing charge data in a timely fashion did reduce costs, but it also resulted in the omission of essential and costly tests. Further studies evaluating the influence of cost-containment measures in the care of actual patients should assess the influence of group dynamics in clinical decision making.

Back to Top | Article Outline


1. Donaldson RM, Joyce CM, Feinstein AR. Effect of restraints on diagnostic approaches to abdominal pain and weight loss. Am J Med. 1986;81:641–7.
2. Griffith CH, Hillson SD, Wilson JF, Rich EC. Internal medicine residency training and patient outcomes. J Gen Intern Med. 1997;12:390–6.
3. Daltroy L, Liang MH. Techniques to improve physicians' use of diagnostic tests: a new conceptual framework. JAMA. 1998;280:2020–7.
4. Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J med. 1990;322:1499–504.
5. Starfield B, Powe NR, Weiner JR, et al. Costs vs quality in different types of primary care settings. JAMA. 1994;272:1903–8.
6. Medical Knowledge Self-Assessment Program VIII. Philadelphia, PA: American College of Physicians, 1989.
7. Feinglass J, Schroeder J, Martin GJ, Wallace W, Lyons J. The relationship of residents' autonomy and use of a teaching hospital's resources. Acad Med. 1991;66:549–52.
8. Taylor DH, Whellan DJ, Sloan FA. Effects of admission to a teaching hospital on the cost and quality of care for Medicare beneficiaries. N Engl J Med. 1999;340:293–9.
9. Larson RL, Christensen C, Franz TM, Abbot AS. Diagnosing groups: the pooling, management, and impact of shared and unshared case information in team-based medical decision making. J Pers Soc Psychol. 1998;75:93–108.
© 2001 Association of American Medical Colleges