Hodgson, Carol S.; Teherani, Arianne; Gough, Harrion G.; Bradley, Pamela; Papadakis, Maxine A.
Defining, measuring, and predicting professionalism in medical students has gained interest and importance during the past five years. The American Board of Internal Medicine’s charter on professionalism states that “professionalism is the basis of medicine’s contract with society.”1 This responsibility toward society was paramount in an earlier study from three medical schools which showed that unprofessional behavior during medical school was associated with later disciplinary action by a state medical board.2 The most important predictive factors were severe levels of irresponsibility and diminished capacity for self-improvement. Anxiety, insecurity, and nervousness manifesting as unprofessional behavior approached significance (P = .06).2 Similar results were found in a smaller study in which irresponsibility, lack of self-improvement, and poor initiative/motivation were predictive of later disciplinary action.3 In another study, examples of irresponsibility (failing to complete required course evaluations and failing to report compliance with immunization requirements) predicted unprofessional behavior during clerkships.4 These studies indicate the importance of unprofessional behavior as a predictor of later problems. Many medical schools now use some method to evaluate professionalism, including observation by faculty, ratings by standardized patients, and completion of professionalism forms on students who display unprofessional behavior.5–8 The best method to assess medial students’ level of professionalism remains unknown.
Eva and colleagues9 examined noncognitive indicators in the medical school admissions process. Their multiple mini-interview (MMI) administered during the admissions interview period was later found to be related to performance on an objective structured clinical examination, whereas other more traditional tools used at admissions (e.g., grade point average [GPA] and personal interviews) were not. One student who scored in the lowest percentile ranking (15th) for the MMI was later cited for unprofessional behavior, although that student had scored in the 76th percentile for GPA and for their simulated tutorial.10
The California Psychological Inventory (CPI) was first developed by Gough11 (p1) in 1948 as a tool to measure personality “for predicting future behavior and understanding prior actions.” The purpose of the CPI is to give a coherent portrait of individuals, in terms of positive psychological qualities and adaptive behavior. In its language, it appeals to everyday universals of description—that is, “folk concepts.” This instrument has been used in numerous venues, including academia, industry, the federal government, and the criminal justice system. More than 2,000 publications cite its use and provide data on the reliability of the instrument.12 The 434-item CPI has 20 folk concept scales. Low scores on these scales indicate the following: responsibility: self-indulgent, undisciplined, not reliable, irresponsible, lacks self-improvement; socialization: resists rules, nonconforming, rebellious, unconventional, cynical; self-control: strong feelings and emotions, problems with undercontrol and impulsivity; communality: sees self as different from others, unconventional, often changeable and moody, lazy; and well-being: concerned about health and/or personal problems, tends to complain about being treated unfairly or inconsiderately, pessimistic, nervous, anxious.11 Three vectors measure higher order constructs. Each vector is measured on a continuum. Vector 2 ranges from norm questioning to norm favoring and is a combination of three scales: socialization and communality minus flexibility. Those scoring low on Vector 2 (rule respecting) can be described as rule testing, nonconforming, self-indulgent, unreliable, and rebellious.11
In 1954, the CPI was first tested as a predictive instrument of medical school success.13 Only the CPI socialization scale differentiated students who graduated from medical school from those who were not admitted. Third-year GPA, overall GPA, and faculty ratings were correlated to the sociability scale. Third-year GPA was also correlated to tolerance and intellectual efficiency. The initial sample was cross-validated with another medical school, using a regression equation with sociability, tolerance, communality, and capacity for status predicting faculty ratings.13 More recently, studies with Thai and Australian medical students used the CPI to examine its usefulness in the admissions process. In the Thai study, the CPI tested the relationship between academic measures of success in medical school. The CPI dominance, flexibility, and socialization scales were positively correlated to GPA, whereas sociability and well-being scales were negatively correlated.14 The CPI scales were also found to correlate with academic achievement in the Australian study. Interviews of matriculates were used to evaluate students’ motivation, cognitive style, interpersonal style, and communication skill in combination with scores on the CPI. No correlations were found between prior academic achievement and the interview scores; however, there were moderate correlations between the interview scores and CPI scales.15 The CPI was also used at application to residency in anesthesia. First-year residents in six American anesthesiology programs were administered the CPI at the beginning of residency. Performance at the end of years one and two was assessed by faculty ratings and correlated with scores on the CPI scales. Higher-performing residents were found to score higher than lower-performing residents on the following CPI scales: dominance, independence, empathy, responsibility, socialization, achievement via conformance, and well-being. High performance was also related to an alpha personality as measured by the CPI. These individuals are typically described as independent, reliable, and self-disciplined. The authors concluded that the CPI is a valuable tool for selecting potential residents with the characteristics desired in their residents.16 The CPI has been used outside of the field of medicine. A case–control study of law enforcement officers showed that officers who had received disciplinary action against them, when compared with nondisciplined officers, scored significantly lower on the CPI responsibility, socialization, and self-control scales.17
The findings from the above studies indicate a relationship between disciplinary action and certain domains of unprofessional behavior in both medical school graduates and law enforcement officers. These results and access to a unique database lead us to test the relationship between specific CPI psychological indicators and unprofessional behavior during medical school. Specifically, do medical school matriculates who demonstrated unprofessional behavior differ from those who did not demonstrate unprofessional behavior during medical school on CPI scale scores for responsibility, socialization, self-control, communality, well-being, and rule respecting?
With institutional review board approval, the study used a case–control descriptive design to examine the relationships between psychological indices as measured by the CPI and measures of professionalism during medical school.
The medical students in this study were from the University of California, San Francisco (UCSF). Information on these graduates is part of existing databases from retrospective case–control studies which examined the association of unprofessional behavior in medical school with later disciplinary action by a state medical board.2,3 The methodology used to select these graduates is previously described.2,3 Data from the UCSF cases and controls were extracted from the graduates’ academic records including their admission, course grades, evaluation narratives, scores from licensing exams, administrative correspondence, and the deans’ letters of recommendation. Data from the academic files were entered into a database and coded by investigators blinded to the case or control status of each subject. At least two investigators determined whether the excerpts described unprofessional behavior (no/yes). The definition of unprofessional behavior was based on previously established criteria.5,18 Types of unprofessional behavior were developed by analyzing the excerpts of negative comments in the medical student records and these were used to determine the appropriate CPI scales for this study.2,3
Psychology faculty at the University of California, Berkeley administered the CPI to UCSF medical students from 1951 to 1970; all medical students matriculating to UCSF during this period were recruited to participate. Data were originally collected on 650 medical students. CPI item data were in the CPI Form 472 format and were scored using the Form 434 scales. Reliability estimates for a normal sample of males completing the CPI are as follows: Cronbach’s alpha coefficient (n = 3,000) and a 25-year test–retest reliability interval (n = 44), respectively, are: responsibility (α = 0.77, r = 0.59); socialization (α = 0.75, r = 0.45); communality (α = 0.74, r = 0.59); well-being (α = 0.84, r = 0.79); and rule respecting (α = 0.76, r = 0.71).11
Data on medical students in the CPI study who were also in the professionalism studies2,3 were provided to the UCSF research staff. The two databases were merged, and all subject identifiers were removed from the database before analyses. Of the graduates for whom professionalism had been previously evaluated on the basis of their academic records (n = 264),3 only 26 also had CPI data available. Therefore, the final data set consisted of 26 subjects, seven of whom demonstrated unprofessional behavior during medical school (cases) and 19 who did not (controls). A priori to conducting this study, to enhance power, unprofessional behavior in medical school replaced disciplinary action by a state medical board as the outcome variable. Descriptive statistics were calculated for the CPI scales (responsibility, socialization, self-control, communality, well-being, and rule respecting). The mean values for this sample (n = 26) were compared with the original sample of 551 male medical students reported in 1996 by Gough and Bradley,11 using a one-sample t test. This was done to test whether the smaller sample in this study was representative of the population of medical students originally tested with the CPI. Pearson product–moment correlation coefficients were calculated between variables. A total score was computed by adding the scores for the six CPI scales (responsibility, socialization, self-control, communality, well-being, and rule respecting). Mean differences between the groups who did and did not demonstrate unprofessional behavior during medical school on raw scores of the CPI scales and the total composite score were tested, using independent t tests.
All but one graduate were male; year of matriculation for the sample ranged from 1960 to 1965. The means and standard deviations for each of the CPI scales are given in Table 1 as are the means for the original 1996 sample of 551 male medical students11 completing the CPI. The CPI scale means of the reference sample (n = 551) compared with the overall means in this sample (n = 26) were not significantly different. Therefore, this smaller sample of 26 graduates is representative of the original larger sample of male medical students.
The graduates’ level of professionalism was significantly correlated to the CPI scales of responsibility (r = 0.53), communality (r = 0.50), well-being (r = 0.46), and rule respecting (r = 0.65). There were significant differences between the graduates in the unprofessional behavior group versus those who did not demonstrate unprofessional behavior (see Table 1) on responsibility, communality, well-being, rule respecting, and on the total score. There were no significant differences between the groups on socialization or self-control, the variables found to be significant for disciplined law enforcement officers.
This study indicates that the scores of the CPI, which was taken at matriculation to medical school, are related to a measure of professionalism during medical school. The psychological indices of the CPI scales differed by level of unprofessional behavior, which leads one to wonder whether personality measures should be considered during the admissions process to medical school. The CPI has been used to assess personality characteristics important to the specialty of anesthesiology, which indicates that some specialties are looking for more than the Medical Student Performance Evaluation, United States Medical Licensing Examination exam scores, letters of recommendation, and internal interviews to help them sort through the applications of medical students who want entry into their profession. If these indices measure the state of the examinee rather than his or her psychological trait, it would be unreasonable to expect these measures to predict professional behavior beyond the short term. Although the CPI scales are stable over time (see above 25-year test–retest data), they have not been shown to be stable through one’s lifetime.11
The small sample size of this study precluded testing whether scores on the CPI would differ for those graduates who were disciplined by a state medical board versus those who were not. Although sample size is an important limitation of this study, the means for each CPI scale and vector used were not significantly different from the original sample of 551 male medical students, indicating that the CPI results are consistent with earlier findings.
Medical schools are struggling to find the best way to assess medical students’ level of professionalism during their tenure in medical school. How do we remediate unprofessional behavior and what do we do if we can’t remediate? Will medical schools eventually be taken to task if they allow students who exhibit unprofessional behavior to graduate? Is it time for medical schools to consider indicators beyond traditional academic performance, the interview, letters of recommendation, and life experiences to play a part in the admissions process? We hope that the results of this unique study will prompt discussion and further research into these issues.
In conclusion, the results of this study are consistent with other findings in which general unprofessional behavior during medical school can be further characterized to domains of poor reliability and responsibility, lack of self-improvement and adaptability, and poor initiative and motivation.2,3 Scores from the CPI, an established instrument, correlate with unprofessional behavior during medical school, providing further evidence of construct validity of these measures and raising the question of whether there is a role for psychological measures at application to medical school or residency.
1ABIM Foundation. Medical professionalism in the new millennium: a physician charter. Ann Intern Med. 2002;136:243–246.
2Papadakis MA, Teherani A, Banach MA, et al. Unprofessional behavior in medical school and subsequent disciplinary action by state medical boards. N Engl J Med. 2005;353: 35–44.
3Teherani A, Hodgson CS, Banach M, Papadakis MA. Domains of unprofessional behavior during medical school associated with future disciplinary action by a state medical board. Acad Med. 2005;8:S17–S20.
4Stern DT, Frohna A, Gruppen LD. The prediction of professional behavior. Med Educ. 2005;39:75–82.
5Papadakis MA, Osborn EH, Cooke M, Healy K. A strategy for the detection and evaluation of unprofessional behavior in medical students. University of California, San Francisco School of Medicine Clinical Clerkships Operation Committee. Acad Med. 1999;74:980–990.
6Papadakis MA, Loeser H, Healy K. Early detection and evaluation of professionalism deficiencies in medical students: one school’s approach. Acad Med. 2001;76:1100–1106.
7Bennett AJ, Arnold LM, Welge JA. Use of standardized patients during a psychiatry clerkship. Acad Psychiatry. 2006;30:185–190.
8Bennett AJ, Roman B, Arnold LM, Kay J, Goldenhar LM. Professionalism deficits among medical students: models of identification and intervention. Acad Psychiatry. 2005;29:426–432.
9Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: the multiple mini-interview. Med Educ. 2004;38:314–326.
10Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med. 2004;79:S40–S42.
11Gough HG, Bradley P. CPI Manual. 3rd ed. Palo Alto, Calif: Consulting Psychologists Press; 1996.
12Gough HG. Comprehensive bibliography of the CPI assessment: 1948–2002. Palo Alto, Calif: Consulting Psychologists Press; 2003.
13Gough HG, Hall WB. Prediction of performance in medical school from the California Psychological Inventory. J Appl Psychol. 1964;48:218–226.
14Chan-Ob T, Boonyanaruthee V. Medical student selection: which matriculation scores and personality factors are important? J Med Assoc Thai. 1999;82:604–610.
15Tutton PJ. Medical school entrants: semi-structured interview ratings, prior scholastic achievement and personality profiles. Med Educ. 1993;27:328–336.
16McDonald JS, Lingam RP, Gupta B, Jacoby J, Gough HG, Bradley P. Psychologic testing as an aid to selection of residents in anesthesiology. Anesth Analg. 1994;78:542–547.
17Muchinsky PM, Nelson-Gray RO. Prediction of dysfunctional job behaviors among law enforcement officers. J Appl Psychol. 1998;83:904–912.
18Papadakis MA, Hodgson CS, Teherani A, Kohatsu ND. Unprofessional behavior in medical school is associated with subsequent disciplinary action by a state medical board. Acad Med. 2004;79:244–249.