The goals of medical education are to train knowledgeable and professional physicians, advance the science of medicine, and promote public health. How best to achieve these goals is the subject of ongoing debate. To clarify the essential skills to be acquired during graduate medical training, the Accreditation Council for Graduate Medical Education (ACGME) has endorsed six general competencies for physicians in training: medical knowledge, professionalism, patient care, systems-based practice, practice-based learning and improvement, and interpersonal and communication skills.1,2 These competencies have served as guidelines for the development and assessment of residency education programs since their introduction by the ACGME in 1999. Although they identify the desired outcomes of residency training, how to effectively teach and measure the development of these qualities has yet to be fully defined.
One proven tool to assess competency in medical knowledge during residency training is the Internal Medicine In-Training Examination (ITE),1,3,4 which has been administered annually to residents at the majority of U.S. internal medicine training programs since 1988. This examination was originally developed by the American College of Physicians, the Association of Program Directors in Internal Medicine, and the Association of Professors of Medicine as a self-evaluation instrument for internal medicine residents at the midpoint of their residency training, although thousands of residents at each year of training now sit for the examination each year.5,6 ITE scores have been shown to be internally consistent, reliable, and accurate measures of medical knowledge of internal medicine.5 Results from the ITE correlate closely with American Board of Internal Medicine certification examination scores7–11 and may, in fact, provide a more accurate estimate of a trainee’s actual knowledge because most examinees do not typically prepare specifically for the ITE.5 Although several authors have reported trends in ITE performance over the last two decades, little is known about what factors contribute to residents’ test performance or how it relates to competency in other domains.5,6,12–15
Professionalism, another of the six core competencies, is considered a fundamental trait of the physician. This attribute requires compassion, integrity, honesty, and respect for patient autonomy. Empathy is considered a central component of professionalism and refers to the physician’s ability to understand how a patient’s feelings and experiences influence their perspective and to convey sentiments of support and compassion. Patients closely associate empathy with professionalism,16–19 and empathy has been termed “the foundation of the patient–physician relationship.”20 Several well-validated instruments have been developed to assess empathy, including the Interpersonal Reactivity Index (IRI) developed by Davis.21 Although this aspect of professionalism has been shown to be associated with competency in patient care,22,23 its relationship to medical knowledge and other domains of competency is poorly understood.
Traditional resident evaluation systems based on standardized knowledge tests and assessment of performance by supervising physicians do not necessarily measure the full breadth of clinical competence3 or distinguish how competency in one domain relates to competency in another. As such, although the competencies specify the qualities medical graduates should possess, relationships among the domains of competency have gone largely unexplored. To examine two domains of competency measurable by well-validated instruments as well as the possible relationship between them, we performed a prospective, longitudinal study of residents’ medical knowledge and professionalism in an academic internal medicine training program.
All 95 categorical and 49 preliminary postgraduate year one (PGY-1) internal medicine trainees in academic years 2003–2004 and 2004–2005 at the Mayo Clinic College of Medicine in Rochester, Minnesota were invited to participate in the Mayo Internal Medicine Well-Being Study (Mayo IMWELL Study), a broad study of residents’ well-being from which we extracted specific measures for our more focused study. Residents in this program attended 86 different U.S. and international medical schools. Residents spent approximately one half of their rotations on in-hospital services with overnight call responsibilities, with the remaining time spent in outpatient and subspecialty consultation rotations. Trainees were invited to participate in this study during their orientation before beginning residency, or by personal telephone call if unable to attend orientation. Participation was elective, and individuals who desired to participate signed written informed consent to be surveyed prospectively and longitudinally every three months throughout their training experience. The consent also gave permission for researchers to access a variety of performance measures contained within resident academic files, including ITE scores. This work was supported by a grant from the Mayo Clinic Department of Medicine. The Mayo Clinic institutional review board approved this study.
Residents were surveyed approximately every three months throughout their training. The present study focuses on data collected through October 2005 for PGY-1 and postgraduate year two (PGY-2). Surveys addressed a variety of topics, including demographic characteristics, current rotation characteristics, quality of life, and personal wellness promotion practices. We used validated survey tools to measure empathy as a surrogate measure of resident competency in the domain of professionalism, as described below. No member of the Mayo Clinic Department of Medicine had access to individual residents’ responses, and nonidentifying numerical codes were used by statisticians to preserve residents’ anonymity when collecting longitudinal data. No member of the department of medicine had access to identifying information on any study item for any participant. ITE scores were linked to the survey time point closest to the date of ITE administration.
The ITE, the content of which has been previously well described,5,6 is administered each October, and results are sent to program directors in December or January. Categorical internal medicine residents at the Mayo Clinic are required to take this exam annually during their three years of training. Because ITE testing in the Mayo program is not required for preliminary residents, the present report focuses only on categorical residents. The ITE is a multiple-choice examination, and scores are reported as the raw percentage of correct answers and the percentile score for each resident’s peer group. For this report, we used the raw scores. ITE scores have well-documented reliability, accuracy, and internal consistency, allowing educational feedback to persons taking the test.5 There is also evidence of a meaningful correlation between ITE performance and passage of future certification examinations,7–11 further supporting the validity of this test as a measure of medical knowledge.
The IRI is a 28-item instrument with four separate seven-item subscales evaluating different dimensions of empathy that are considered independently.21 Each question item is scored on a Likert scale from 0 to 4, where 0 = “does not describe me well” and 4 = “describes me well,” so that the maximum score for each subscale is 28. For this study, we chose to include the IRI subscales measuring cognitive and emotive domains of empathy (abbreviated, respectively, as perspective taking subscale, IRI-PT, and empathic concern subscale, IRI-EC). The cognitive empathy subscale evaluates whether an individual understands the perspective of another person about their circumstances, and the emotive empathy subscale evaluates an individual’s concern for the feelings of another person. The IRI subscales have been shown to be reliable and reproducible measures of sensitivity to the views and feelings of others,21,24 and this tool has been used in a wide variety of research settings, including a number of previous studies of medical students and physicians.25–30
Statistical analyses were conducted using SAS version 8.2 (SAS Inc., Cary, NC). Statistical significance was set at the .05 level. Standard univariate statistics were used to characterize the sample. Changes in outcomes over time were analyzed using the Wilcoxon signed rank test for paired data. Associations between ITE and IRI scores were assessed using Spearman rank correlations. With 55 respondents, this study had 96% power to detect a half-standard-deviation change in paired continuous scores, an effect size considered to represent a clinically meaningful difference between groups.31 This sample size also allows estimates of correlation coefficients within 0.23 of the true value with 95% confidence.
Eighty-seven of 95 eligible residents (92%) signed written consent to participate in this longitudinal study. Of these, 75 residents had both ITE and IRI scores as PGY-1 trainees. The demographic characteristics of these study participants at the time of study entry are presented in Table 1. Fifty-five residents (58%) provided both ITE and IRI data in the fall of both PGY-1 and PGY-2, yielding data for longitudinal analysis (Figure 1). There were no statistically significant differences in age or gender between participants and nonparticipants.
Mean ITE and IRI scores for all participating residents in the fall of PGY-1 and PGY-2 are shown in Table 2. There were no associations between ITE score and demographic factors. Empathy scores were similar to scores previously reported in studies of medical residents.25,29,30,32 Consistent with previous reports,22,32–34 women had higher scores for both cognitive and emotive empathy than men (by 1.1 and 2.4 points, respectively; data not shown). There were no other observed associations between IRI scores and demographic variables.
Among the 55 residents with data available for both PGY-1 and PGY-2, ITE scores increased an average of 8.7 points from year one to year two (P < .0001), demonstrating an increase in medical knowledge acquired through internship (Figure 2). Thirty-six residents (65.5%) had at least a 10% increase in their ITE score, and 16 (29.1%) had a 20% increase. No resident had a 10% decrease in ITE score on longitudinal follow-up (Table 3).
Among these residents, a statistically significant decline of 1.6 points in emotive empathy was observed during the first year of training (P = .0003; Figure 2). Whereas three residents (5.5%) had at least a 10% increase in their emotive empathy score over the first year of training, 21 (38.2%) had at least a 10% decrease in emotive empathy scores, and 10 (18.2%) had a 20% or greater decrease (Table 3). Although a small decline of 0.5 points in mean cognitive empathy scores was also observed among these residents, this difference did not reach statistical significance (P = .4287; Figure 2). Roughly equal percentages of residents experienced increases and decreases in cognitive empathy scores over their first year of training (Table 3). None of these changes were associated with demographic factors.
We next explored the relationship between residents’ medical knowledge and empathy as measured by the ITE and IRI, respectively. Medical knowledge showed no correlation with either cognitive or emotive empathy (all Spearman correlations < 0.1; data not shown). Changes in cognitive and emotive empathy between PGY-1 and PGY-2 were not associated with PGY-1 or PGY-2 ITE score or with changes in ITE score between PGY-1 and PGY-2 (data not shown).
The six core competencies proposed by the ACGME were intended to define the distinct areas of expertise required of all physicians. How competency in one domain relates to competency in another is not well understood. Such interactions are complex phenomena because the six core competencies have both shared and independent features. For example, competency in the domains of medical knowledge and communication skills are requisite for, but do not guarantee, competency in patient care. Accordingly, the ACGME has contended that each of these elements should be evaluated separately, although minimal empiric data are available to defend this premise.
Conceptually, this philosophy of resident training and evaluation is quite different from the traditional view of graduate medical training and evaluation based on the apprentice model. Under this historic system, residents were immersed in a training environment expected to imbue competency in all domains through the process of observation and experience. Evaluation systems were primarily limited to performance on standardized knowledge tests and global evaluations of performance made by supervising physicians and were typically heavily focused on competency in medical knowledge and patient care.35
In this paper, we report selected findings from the Mayo IMWELL Study, a prospective longitudinal study of the relationships between multiple domains of resident competency and personal well-being. Our results support the premise that the core competencies represent separate domains of skill/expertise that develop independently. The absence of a relationship between measures of professionalism (empathy) and medical knowledge in our study implies that these competencies are influenced by separate and perhaps independent aspects of training, because the influence of any single factor affecting both domains would generally be expected to result in significant correlations between them. The observed concurrent increase in medical knowledge and decrease in empathy over the first year of training underscores the point that simply immersing residents in a training environment should not be expected to result in “global competency.” In fact, these data demonstrate that the same environment may simultaneously promote competency in one domain (e.g., medical knowledge) and erode competency in another (e.g., professionalism). These results support the ACGME premise that training programs should devise specific curricula to develop each domain of competency, rather than assume that elements such as professionalism are innate or are routinely acquired through the course of the training experience.
In addition to emphasizing the need for an intentional curriculum for the development of each competency, our findings confirm the importance of measuring each domain of competency separately. Although validated metrics are available to measure competency in some domains, how best to evaluate progress in other areas has yet to be defined. For the domain of medical knowledge, instruments such as the ITE have been developed that allow assessment of residents’ gains from the beginning of the training process. The development of validated tools to evaluate competency across other domains is needed. Standardized tests may be an effective way to measure medical knowledge; however, other methods such as standardized patients, portfolios, evaluation of residents by their patients, and videotaped encounters that can be objectively reviewed may be necessary to evaluate competency in other domains.1,3
The observed decrease in empathy among the residents in this study over the first year of training is consistent with the results of several other studies showing that humanistic attitudes decline throughout medical school and residency.25,30,36–39 Several studies have suggested this decline in empathy may be related to personal distress during the training experience,40–42 implying that efforts to teach and promote professionalism must consider not only the patient–physician relationship but also physicians’ personal well-being. In addition to longitudinal evaluation of resident competency, the Mayo IMWELL Study longitudinally measures resident quality of life, burnout, and depression; its findings will provide insights into relationships between domains of competency and resident distress.
This study has several limitations. First, the generalizability of these findings from a single academic medical center to other training programs is unknown. Residents in this study attended a broad range of medical schools and worked in inpatient and outpatient settings characteristic of academic residency training programs. Although variations in the absolute levels of empathy and medical knowledge exist among residents in different training programs, we suspect the relationship we observed between these variables for the residents in this study is similar to the relationship between variables noted in other resident samples. Residents’ mean empathy scores seen in this study were similar to those found in other samples of medical residents,25,29,30,32 suggesting that our findings reflect general rather than local phenomena. Our findings are also consistent with previous studies of medical students suggesting that empathy may be independent of performance on standardized tests.33,43,44
Second, this study does not provide information on whether the observed changes in residents’ medical knowledge and empathy, or the lack of a relationship between them, persist beyond October of PGY-2. Third, although the IRI is a validated measure of empathy, it remains possible that this survey tool does not accurately capture resident behavior. Future work incorporating observed resident practices is necessary to fully evaluate behavioral aspects of the ACGME core competencies.
In conclusion, this prospective longitudinal study supports the ACGME’s contention that the different domains of physician competency should be developed and evaluated separately. Although ITE scores serve as a useful measure of medical knowledge, they do not seem to be useful as a surrogate marker for identifying residents struggling in other domains of competency or to be necessarily valid as a global measure of physician quality. Perhaps even more important, our findings suggest that aspects of the training experience that promote development of one competency may simultaneously erode other competencies. This observation underscores the need for training programs to develop curricula to promote each of the core competencies individually rather than assuming that the traditional system of training effectively imparts all of the qualities desired of the competent clinician. Additional prospective studies are needed to identify the best methods to develop each competency and to identify accurate metrics to measure progress in the attainment of each skill.
This study was supported by a Medicine Innovation Development and Advancement System (MIDAS) grant from the Mayo Clinic Department of Medicine.
1 Goroll AH, Sirlo C, Duffy FD, et al. A new model for accreditation of residency programs in internal medicine. Ann Intern Med. 2004;140:902–909.
2 Accreditation Council for Graduate Medical Education. Outcome Project. Available at: (http://www.acgme.org/Outcome
). Accessed February 5, 2007.
3 Holmboe ES, Hawkins RE. Methods for evaluating the clinical competence of residents in internal medicine: a review. Ann Intern Med. 1998;129:42–48.
4 Hawkins RE, Sumption KF, Gaglione MM, Holmboe ES. The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance. Am J Med. 1999;106:206–210.
5 Garibaldi RA, Trontell MC, Waxman H, et al. The in-training examination in internal medicine. Ann Intern Med. 1994;121:117–123.
6 Garibaldi RA, Subhiyah R, Moore ME, Waxman H. The in-training examination in internal medicine: an analysis of resident performance over time. Ann Intern Med. 2002;137:505–510.
7 O’Dell JR. In-training examination in internal medicine. Ann Intern Med. 1995;122:73–74.
8 Grossman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the in-training examination for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med. 1992;7:63–67.
9 Wolfsthal SD, Hooper FJ. In-training examination in internal medicine. Ann Intern Med. 1995;122:73–74.
10 Waxman H, Braunstein G, Dantzker D, et al. Performance on the internal medicine second-year residency in-training examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med. 1994;9:692–694.
11 Rollins LK, Martindale JR, Edmond M, Manser T, Scheld WM. Predicting pass rates on the American Board of Internal Medicine certifying examination. J Gen Intern Med. 1998;13:414–416.
12 Waxman HS. Workforce reform, international medical graduates, and the in-training examination. Ann Intern Med. 1997;126:803–805.
13 Norcini JJ, Shea JA, Webster GD, Benson JA Jr. Predictors of the performance of foreign medical graduates on the 1982 certifying examination in internal medicine. JAMA. 1986;256:3367–3370.
14 Waxman HS, Garibaldi RA, Subhiyah RG. Performance of U.S. and international medical graduates on the 1995 internal medicine in-training examination. Ann Intern Med. 1996;125:158.
15 Norcini JJ, Grosso LJ, Shea JA, Webster GD. The relationship between features of residency training and ABIM certifying examination performance. J Gen Intern Med. 1987;2:330–336.
16 Katic M, Budak A, Ivankovic D, et al. Patients’ views on the professional behavior of family physicians. Fam Pract. 2001;18:42–47.
17 Wensing M, Jung HP, Mainz J, Olesen F, Grol R. A systematic review of the literature on patient priorities for general practice care. Part 1: Description of the research domain. Soc Sci Med. 1998;47:1573–1588.
18 Beckman HB, Markakis KM, Suchman AL, Frankel RM. The doctor-patient relationship and malpractice. Lessons from plaintiff depositions. Arch Intern Med. 1994;154:1365–1370.
19 Falvo DR, Smith JK. Assessing residents’ behavioral science skills: patients’ views of physician-patient interaction. J Fam Pract. 1983;17:479–483.
20 Rosenow EC. Recertifying in the art of medicine: what I would tell young physicians. Mayo Clin Proc. 2000;75:865–868.
21 Davis M. A multidimensional approach to individual differences in empathy. JSAS Catalog of Selected Documents in Psychology. 1980;10:85.
22 Hojat M, Gonnella JS, Nasca TJ, Mangione S, Vergare M, Magee M. Physician empathy: definition, components, measurement, and relationship to gender and specialty. Am J Psychiatry. 2002;159:1563–1569.
23 Colliver JA, Willis MS, Robbs RS, Cohen DS, Swartz MH. Assessment of empathy in a standardized-patient examination. Teach Learn Med. 1998;10:8–11.
24 Davis M. Measuring individual differences in empathy: evidence for a multi-dimensional approach. J Pers Soc Psychol. 1983;44:113–126.
25 Bellini LM, Baime M, Shea JA. Variation of mood and empathy during internship. JAMA. 2002;287:3143–3146.
26 Evans B, Stanley R, Burrows G. Measuring medical students empathy skills. Br J Med Psychol. 1993;66:121–133.
27 Evans B, Stanley R, Coman G, Burrows G. Psychological tests to measure the effects of medical education on students’ interpersonal skills. Med Educ. 1989;23:492–497.
28 Coman G, Evans B, Stanley R. Scores on the Interpersonal Reactivity Index: a sample of Australian medical students. Psychol Rep. 1988;62:943–945.
29 Rosen IM, Gimotty PA, Shea JA, Bellini LM. Evolution of sleep quantity, sleep deprivation, mood disturbances, empathy, and burnout among interns. Acad Med. 2006;81:82–85.
30 Bellini LM, Shea JA. Mood change and empathy decline persist during three years of internal medicine training. Acad Med. 2005;80:164–167.
31 Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life. The remarkable universality of a half a standard deviation. Med Care. 2003;41:582–592.
32 Shanafelt TD, West CP, Zhao X, et al. Relationship between increased personal well-being and enhanced empathy among internal medicine residents. J Gen Intern Med. 2005;20:559–564.
33 Hojat M, Gonnella JS, Mangione S, et al. Empathy in medical students as related to academic performance, clinical competence and gender. Med Educ. 2002;36:522–527.
34 Bylund CL, Makoul G. Empathic communication and gender in the physician-patient encounter. Patient Educ Couns. 2002;48:207–216.
35 Ginsburg S, Regehr G, Hatala R, et al. Context, conflict, and resolution: a new conceptual framework for evaluating professionalism. Acad Med. 2000;75:S6–S11.
36 Hojat M, Mangione S, Nasca TJ, et al. An empirical study of decline in empathy in medical school. Med Educ. 2004;38:934–941.
37 Collier VU, McCue JD, Markus A, Smith L. Stress in medical residency: status quo after a decade of reform? Ann Intern Med. 2002;136:384–390.
38 Feudtner C, Christakis DA, Christakis NA. Do clinical clerks suffer ethical erosion? Students’ perceptions of their ethical environment and personal development. Acad Med. 1994;69:670–679.
39 Self DJ, Schrader DE, Baldwin DC Jr, Wolinsky FD. The moral development of medical students: a pilot study of the possible influences of medical education. Med Educ. 1993;27:26–34.
40 Kassirer JP. Doctor discontent. N Engl J Med. 1998;339:1543–1545.
41 Cohen JJ. Heeding the plea to deal with resident stress. Ann Intern Med. 2002;136:394–395.
42 Thomas NK. Resident burnout. JAMA. 2004;292:2880–2889.
43 Stern DT, Frohna AZ, Gruppen LD. The prediction of professional behavior. Med Educ. 2005;39:75–82.
44 Diseker RA, Michielutte R. An analysis of empathy in medical students before and following clinical experience. J Med Educ. 1981;56:1004–1010.
© 2007 Association of American Medical Colleges
This article has been cited