Murden, Robert A.; Way, David P.; Hudson, Andy; Westman, Judith A.
Recently, there has been a renewed emphasis on professionalism in U.S. medical schools.1–4 A small percentage of graduating medical students may have sufficient general medical knowledge, but insufficient interviewing and patient communication skills to be effective physicians,5 a problem which has been demonstrated in practicing physicians.6 Accordingly, the United States Medical Licensing Examination (USMLE™) is instituting a clinical skills examination in 2005.5 The Accreditation Council for Graduate Medical Education will require evaluation of residents in communication and professionalism by 2007.7
Despite this renewed emphasis on professionalism, early identification of medical students having problems with professionalism has not been well studied. Editorials in medical journals have called for having personal characteristics8 and evaluations of moral reasoning9 included in the medical school selection process. In a few studies, noncognitive characteristics have been shown to be predictors of both preclinical and clinical performance.10–13 These studies compared personality profiles,10,11,13 or nonverbal behaviors during clinical encounters,12 which were obtained solely for purposes of the studies. None of these studies compared early, routine medical school evaluations of noncognitive characteristics with subsequent clinical performance.
Students identified during the clinical years as having deficiencies in noncognitive areas or problems with professionalism, are sometimes flagged within the first two years of medical school.14,15 While students with noncognitive or professionalism problems of significant magnitude are rare,14,15 concrete methods for identifying and managing such students early in their medical education should be formulated.
Over 20 years ago, in order to provide early exposure to important clinical skills, the Ohio State University (OSU) initiated a first-quarter course in physician–patient communications. This doctor–patient relationship (DPR) course consists of eight three-hour sessions, seven of which are small-group meetings. Small groups consist of one physician and one nonphysician preceptor, and 12 to 14 students. Two sessions use videotaped practice interviews of simulated patients that are critiqued by the preceptors. Other sessions consist of role-playing, patient presentations, and educational activities designed to teach and assess important aspects of DPR and communication skills. At the end of the course, students receive a summative evaluation of their performance with a narrative about strengths and weaknesses.
Over the years, DPR course student assessments revealed some infrequent, yet consistent deficiencies in student performance. These deficiencies consistently fell into one of four categories, formulated by the course director and coordinator:
1. Extreme shyness: Student rarely participated in group discussions, lacked eye contact with the patient, was withdrawn, isolated, and/or cold during interviews.
2. Poor process skills: Student could not gather data in an organized and complete fashion, consistently failed to elicit relevant information, or clarify and reiterate patient concerns and problems.
3. Paternalism: Student exhibited paternalistic and controlling behavior during interviews, often focused the interview on self rather than patient, did not listen to, respond empathically to, or establish rapport with patients.
4. Negative attitude toward course: Student complained or demonstrated a negative attitude toward DPR activities, rarely volunteered to participate and was reluctant when asked, appeared disinterested in the course material, and was intolerant of others.
The purpose of this study was to determine whether identification of these deficiencies in the DPR course might predict poor performance on third-year core clerkships.
Subjects were first-year students from the entering classes of 1996, 1997, and 1998 (n = 630). Each year, preceptors were sent a cover letter and a list of the four deficiencies during the last week of the DPR course. The cover letter for preceptors defined the purpose of the study as research, without implications for student grading or the student record. Both group facilitators (physician and nonphysician) were asked to jointly identify students whose performance exhibited one or more of the deficiencies.
Institutional Review Board approval was obtained and additional tracking information was collected from the student records office. Final grades for all courses, and variables from the admissions record were gathered from student academic records. Grades received during the third-year clerkships, as determined by subjective checklists and multiple choice examinations, were assigned a numeric value then converted to a percentage score for the seven core clerkships: ambulatory medicine, internal medicine, neurology, obstetrics–gynecology, pediatrics, psychiatry, and surgery.
The OSU Medical School computes a preadmissions “predictor score” for every applicant. This score is a composite score computed by regressing the Medical College Admission Test (MCAT) subscales, undergraduate science grade point average, and undergraduate college average MCAT scores on USMLE Step 1.16 The score is converted to a percentile rank and used in the admissions process. Each student identified as having a deficiency was matched to another student from the same DPR group based on predictor score (i.e., with the same academic potential) to create a comparison group without any deficiencies. Students who had withdrawn from school or had not yet completed the clerkships were dropped from the study. The paternalistic and poor-attitude deficiency types were combined into one group because of small numbers. This combined group can be thought of as those who had negative attitudes. Average scores for the two groups are shown in Table 1.
A two-way analysis of variance (ANOVA) with one repeated factor was used to test the hypothesis: Are students identified in Year 1 as having professionalism deficiencies significantly different on clerkship performance from a control group matched by general academic ability? The clerkship grades were used as a metric of clinical skills performance, part of which is determined by the ability to interact in a professional manner with patients. Once a significant difference between the two groups was observed with the omnibus test, posthoc paired t-tests were used to determine the deficiency type contributing the most to the observed difference (see Figure 1).
With a 100% survey response rate, a total of 42 students (6.7%) were identified as having 44 specific deficiencies over the three years of the study (two students were identified as having deficiencies in two areas). Four students were dropped from the study because they had either withdrawn or had not yet completed the clerkships. The remaining 38 students graduated by completion of the study.
A matched sample student was identified for every deficient student; however, predictor code scores were not identical. While in some cases the matching student had a higher or lower predictor code, the mean predictor code across groups was nearly identical (see Table 1).
The results of the two-way ANOVA indicate a significant difference between the deficiency group and their controls on third-year clerkship performance (F = 7.45, df = 1/35, p < .05). Posthoc tests reveal that the primary source of this main effect came from the third deficiency type: the inability to establish rapport with patients because of paternalistic behavior, self-centered behavior, or poor attitude towards DPR skills. A median box and whisker plot shows the difference in distributions for each deficiency type (see Figure 1).
Medical students who demonstrate deficiencies in patient communication skills and professionalism early in medical school tend to continue to have difficulties in these areas in the clinically intense third-year clerkships. The deficiency in ability to establish patient rapport is the most predictive of poor clinical performance in the third-year clerkships. It has been shown previously that shyness, poor eye contact, lack of smiling and touching, correlate negatively with clinical evaluations in pediatrics.12 However, in this study, students identified early with shyness seemed to have for the most part overcome the problem with practice. Students having early problems with process skills received lower, but not significantly so, clerkship grades. Thus, these deficiencies were also overcome between the first and third year, which has been previously shown.17,18
Among the identified deficiencies that could be detected in an introductory doctor–patient relationship class, attitudinal issues are the strongest predictor of poor performance in medical school. Such deficiencies have been successfully approached in the past, however. A New Mexico study described how a systematic awareness of professionalism issues in first- and second-year students, with a standard evaluation form, led to identification and remediation.14 Although time and resource consuming, these interventions seem superior to the alternative of allowing students with poor attitudes and professionalism to graduate without attempts at remediation. In addition, appeals have been made to include tests such as moral reasoning among admissions committee evaluations.9
Much of this discussion presupposes that good performance in medical school is indicative of good clinical skills. This may not always be the case as it has been shown that subjective evaluations of clinical knowledge do not necessary correlate with pen-and-paper grades.19 Some clinical faculty speculate that students with charismatic personalities and strong verbal skills are often graded higher than deserved, and students lacking such characteristics are often graded lower. Although this possibility is a potential confounder to our data, multiple previous studies have shown that performance in medical school correlates with subsequent performance as a house officer.20–24 Another potential confounder is the possibility that some preceptors were reluctant to identify students who had these characteristics. This could have skewed the study; however, the 6.7% of all students identified as having problems in this study was consistent with a previous study that found 6% of students having similar deficiencies.14
In summary, failure to establish patient rapport due to poor attitudes or paternalism in a first-year DPR course is predictive of poor performance in the third-year core clerkships. Shy students tend to overcome their shyness and students with initial poor processing skills seem to improve over time, presumably thorough the normal teaching and assessment process. Routine, aggressive surveillance for attitude problems, as early as the admission process, with subsequent remediation for those accepted to medical school is recommended. It seems clear that a strong focus on communication and professionalism, with an emphasis on DPR skills, is an increasingly important part of the medical school curriculum, if we are to graduate competent, professional physicians.
1. Ludmerer K. Instilling professionalism in medical education. JAMA. 1999;282:881–2.
2. Wear D, Castellani B. The development of professionalism: curriculum matters. Acad Med. 2000;75:602–11.
3. Rothman D. Medical professionalism: focusing on the real issues. N Engl J Med. 2000;342:1284–6.
4. Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136:243–6.
5. USMLE Clinical Skills Examination. Fact Sheet NBME Exam. 2002;49:6.
6. Caulford P, Lamb S, Kaigas T, Hanna E, Norman G, Davis D. Physician incompetence; specific problems and predictors. Acad Med. 1994;69:S16–8.
7. Klein E, Jackson J, Kratz L, et al. Teaching professionalism to residents. Acad Med. 2003;78:26–34.
8. Cohen J. Our compact with tomorrow's doctors. Acad Med. 2002;77:475–80.
9. Self D, Baldwin D. Should moral reasoning serve as a criterion for student and resident selection? Clin Orthop Relat Res. 2000;378:115–23.
10. Peng R, Khaw H, Edariah A. Personality and performance of pre-clinical medical students. Med Educ. 1995;29:283–8.
11. Hojat M, Robeson M, Damjanov I, Veloski J, Glaser K, Gonnella J. Students’ psychosocial characteristics as predictors of academic performance in medical school. Acad Med. 1993;68:635–7.
12. Rosenblum N, Wetzel M, Platt O, Daniels S, Crawford J, Rosenthal R. Predicting medical student success in a clinical clerkship by rating students’ nonverbal behavior. Arch Pediatr Adolesc Med. 1994;148:213–9.
13. Shen S, Comrey A. Predicting medical students’ academic performances by their cognitive abilities and personality characteristics. Acad Med. 1997;72:781–6.
14. Phelan S, Obenshain S, Galey W. Evaluation of the noncognitive professional traits of medical students. Acad Med. 1993;68:799–803.
15. Papadakis M, Loeser H, Healy K. Early detection and evaluation of professional deficiencies in medical students: one school's approach. Acad Med. 2001;76:1100–6.
16. Basco WT Jr., Way DP, Gilbert GE, Hudson A. Undergraduate institutional MCAT scores as predictors of USMLE Step I performance. Acad Med. 2001;77(10 suppl):S13–16.
17. Humphris G, Kaney S. Assessing the development of communication skills in undergraduate medical students. Med Educ. 2001;35:225–31.
18. Klamen D, Williams R. The effects of medical education on students’ patient satisfaction ratings. Acad Med. 1997;72:57–61.
19. Awad SS, Liscum KR, Aoki N, Awad SH, Berger DH. Does the subjective evaluation of medical student surgical knowledge correlate with written and oral exam performance? J Surg Res. 2002;104:36–9.
20. Hunt D, Scott C, Phillips T, Yergan J, Greig L. Performance of residents who had academic difficulties in medical school. J Med Educ. 1987;62:170–6.
21. Gunzburger L, Frazier R, Yang L, Rainey M, Wronski T. Premedical and medical school performance in predicting first-year residency performance. J Med Educ. 1987;62:379–84.
22. Hojat M, Gonnella J, Veloski J, Erdmann J. Is the glass half full or half empty? A reexamination of the associations between assessment measures during medical school and clinical competence after graduation. Acad Med. 1993;68:S69–76.
23. Smith S. Correlations between graduates’ performances as first-year residents and their performances as medical students. Acad Med. 1993;68:633–4.
24. Elam C, Johnson M, Haist S. Preadmission and medical school performances of outstanding internal medicine interns. Acad Med. 1994;69:691.