Secondary Logo

Share this article on:

Is Reflective Ability Associated With Professionalism Lapses During Medical School?

Hoffman, Leslie A., PhD; Shew, Ronald L., PhD; Vu, T. Robert, MD; Brokaw, James J., PhD; Frankel, Richard M., PhD

doi: 10.1097/ACM.0000000000001094
Research Reports

Purpose Recently, many have argued that learning to reflect on one’s experiences is a critical component of professional identity formation and of professionalism. However, little empirical evidence exists to support this claim. This study explored the association between reflective ability and professionalism lapses among medical students.

Method The authors conducted a retrospective case–control study of all students who matriculated at Indiana University School of Medicine from 2001 to 2009. The case group (n = 70) included those students who had been cited for a professionalism lapse during medical school; the students in the control group (n = 230) were randomly selected from the students who had not been cited for a professionalism lapse. Students’ professionalism journal entries were scored using a validated rubric to assess reflective ability. Mean reflection scores were compared across groups using t tests, and logistic regression analysis was used to assess the relationship between reflective ability and professionalism lapses.

Results Reflection scores for students in the case group (2.46 ± 1.05) were significantly lower than those for students in the control group (2.82 ± 0.83) (P = .01). A lower reflection score was associated with an increased likelihood that the student had been cited for a professionalism lapse (odds ratio = 1.56; P < .01).

Conclusions This study revealed a significant relationship between reflective ability and professionalism, although further study is needed to draw any conclusions regarding causation. These findings provide quantitative evidence to support current anecdotal claims about the relationship between reflection and professionalism.

L.A. Hoffman is assistant professor, Department of Anatomy and Cell Biology, Indiana University School of Medicine, Fort Wayne, Indiana.

R.L. Shew is senior lecturer, Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, Indiana.

T.R. Vu is associate professor of clinical medicine and associate director of the medicine clerkship, Department of Medicine, Indiana University School of Medicine, Indianapolis, Indiana.

J.J. Brokaw is associate professor, vice chair for education, and director, Indiana University Center for Anatomical Sciences Education, Department of Anatomy and Cell Biology, Indiana University School of Medicine, Indianapolis, Indiana.

R.M. Frankel is professor of medicine, Department of Medicine, and director, Walther Program in Palliative Care Research and Education, Simon Cancer Center, Indiana University School of Medicine, Indianapolis, Indiana.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: The Indiana University institutional review board deemed the research protocol exempt from full board review on May 16, 2012 (protocol number 1205008665).

Previous presentations: Preliminary results of this study were presented at the Central Group on Educational Affairs spring meeting, March 2014, Cleveland, Ohio.

Correspondence should be addressed to Leslie A. Hoffman, Indiana University School of Medicine, 2101 E. Coliseum Blvd., Fort Wayne, IN 46805; telephone: (260) 481-0510; e-mail: lesahoff@iupui.edu.

Reflection has been widely cited as an essential component of professionalism in medicine and other health care professions,1–5 and it has been proposed as a tool to help students learn to distinguish between positive and negative role models and to reconcile the conflicting messages from the formal, informal, and hidden curricula.6 Although many have described the activities or processes that constitute reflection, in this study, we consider it to be an exploration and appraisal of one’s own and others’ experiences for the purpose of clarifying or creating meaning.7 Reflection is a process by which one’s thoughts are “turned back” to an event or experience so it can be analyzed and interpreted to gain a new or better understanding. The insights gained during the reflective process then can be applied to similar situations in the future with the goal of improving outcomes.8 The stimulus for reflection is often some “disorienting dilemma” that causes one to question one’s knowledge and assumptions about a situation.9 The discrepancy between one’s idealized notion of medicine and the reality of medical practice provides such a stimulus for medical students, and reflecting on this dichotomy often can help them navigate among their experiences and ultimately accept or reject elements of the hidden curriculum.10 , 11

Reflective writing as a tool to promote and assess professionalism is growing in popularity among medical schools. A 2008 survey found that 35% of internal medicine (IM) clerkship directors required students to engage in reflective writing during their clerkship, indicating a belief that the practice helped students to develop communication skills and professionalism. Another 16% indicated that they planned to implement reflective writing curricula within the next two years.12 Indiana University School of Medicine (IUSM) introduced reflective writing into the IM clerkship curriculum in 2004. The goal of the reflective writing assignment, which took the form of an online professionalism journal, was to heighten students’ awareness of professionalism issues by encouraging them to reflect on and learn from their experiences.

Although many medical schools have introduced reflective writing curricula to foster students’ professional development, the evidence to support this practice is largely anecdotal; no studies have provided empirical evidence that reflection enhances students’ professionalism or, more important, that a lack of reflection increases the risk of professionalism lapses. The purpose of this study was to examine the relationships between reflection and professionalism using a case–control study design that compared the reflective ability of students who had been cited for professionalism lapses and those who had not. We hypothesized that students who had been cited for professionalism lapses would have lower reflection scores than those who had not been cited for such lapses.

Back to Top | Article Outline

Method

Evaluation of professionalism at IUSM

At IUSM, professionalism and other noncognitive aspects of medical education (e.g., teamwork and communication skills) are part of a formal competency-based curriculum.13 As part of this system, the Student Promotions Committee (SPC) was charged with evaluating competency deficiencies among students and referring them to qualified individuals for remediation. Between 1999 and 2009, 191 students were referred to the SPC for 317 separate competency deficiencies. Deficiencies in professionalism, self-awareness, and moral reasoning were the most frequently cited, accounting for over 55% of all citations (29.3%, 17.7%, and 9.1%, respectively).14 Students who had multiple citations often were cited for deficiencies in more than one of these areas, suggesting that self-awareness and moral reasoning are closely related to professionalism. Current definitions frequently include self-awareness, ethics, and moral judgment as specific domains under the umbrella of professionalism,3 , 15 which prompted us to include all three competency areas—self-awareness, moral reasoning, and professionalism—as “professionalism-related competencies” for the purposes of this study.

The conceptual model of professionalism used by IUSM was adapted from Swick’s3 definition. The SPC compiles and evaluates descriptive comments from course/clerkship directors, clinical preceptors, and other faculty/staff, such that citations for deficiencies are issued by consensus. Prior studies have shown that similar descriptive/qualitative methods of evaluation can achieve a level of validity and reliability that is sufficient for high-stakes decisions.16 , 17 Examples of behaviors that would be deemed unprofessional or lacking in self-awareness or moral reasoning include chronic tardiness or unexcused absences from clinic or other required activities, incomplete assignments, plagiarizing notes in the electronic medical record, or using disrespectful language toward or about a patient. The SPC evaluates all information pertinent to a case and renders a decision about remediation.13 SPC decisions are guided by past precedents to ensure that citations are consistent across similar circumstances.

Back to Top | Article Outline

Study population and sample

Our study population consisted of all medical students who matriculated at IUSM between 2001 and 2009 (N = 2,700). All students included in our study had graduated or were otherwise no longer students at IUSM at the time the data were collected. The case group included all students who had appeared before the SPC for competency deficiencies related to professionalism at any point during medical school. The students in the control group were randomly selected from the entire population of students who had no SPC history. We determined that a total sample size of 300 students was needed to achieve adequate statistical power.18

Back to Top | Article Outline

Assessment of reflective ability

The primary source for determining students’ reflective ability was the professionalism journal entries they submitted during their eight-week IM clerkship. Students were asked to reflect on an experience during the clerkship that taught them something about professionalism, taking into account whether the behavior or action they observed was effective, how they might have responded in a similar situation, and what they learned from the experience. The IM clerkship director granted us permission to use these journal entries on the condition that no current students were included in the study. Prior to any data analysis, we deidentified all journal entries and assigned each an ID number to allow us to later match the journal entries back to a case or control student.

We measured students’ reflective ability using a validated scoring rubric19 that incorporated components of reflection from the models described by Boud and colleagues,20 Mezirow,9 , 21 and Schön.5 The rubric we used included seven levels of reflection (see Table 1). We selected this rubric because the scoring criteria were based on the most widely accepted theories of the reflective process and because its validity has been confirmed by several studies.22–25 We obtained permission from one of the rubric’s authors to use it in this study.26

Table 1

Table 1

Back to Top | Article Outline

Rater training and reflection scoring

Two authors (L.A.H., R.L.S.) were trained by one of the rubric’s authors according to its published guidelines.19 Those two authors scored 5 journal entries together, followed by 10 entries independently, at which point we calculated interrater reliability (IRR) using Cohen weighted kappa.27 A weighted kappa measurement was selected for this study to allow for scaled degrees of agreement, such that a small discrepancy in scoring received greater weight than a large discrepancy.

Once we achieved an IRR of at least 0.8, one of the raters (L.A.H.) scored 559 journal entries written by the 300 students in the sample. If a student submitted multiple journal entries, only the first two entries (as determined by entry date and timestamp) were scored. To ensure reliability, and to prevent drift and decay from the rubric criteria, we assessed IRR after every 65 entries were scored. The secondary rater (R.L.S.) scored the last 15 entries, and we once again calculated IRR. Of the 559 journal entries, 135 were scored by both raters, and overall IRR was 0.84. The distribution of reflection scores is presented in Table 2.

Table 2

Table 2

Back to Top | Article Outline

Statistical analysis

All statistical analyses were conducted using SPSS Version 22 (IBM Corp., Armonk, New York). In addition to calculating reflection scores, we also examined the gender distribution and average age at matriculation for students in the case and control groups. We included gender because a previous analysis at IUSM found that males were significantly more likely to have been cited for competency-related deficiencies.14 We included age at matriculation because a recent study found that students aged 31 or older were significantly more likely to have academic difficulties in medical school,28 and we were interested in whether this group of students had been cited for professionalism deficiencies as well. We conducted t tests using independent samples to compare reflection scores and average age at matriculation between the two groups. If a student had two scored journal entries, we included only the higher score in the statistical analysis. The distribution of students’ ages at matriculation was highly skewed, so we used a log transformation to achieve a normal distribution required to conduct a t test. We compared the gender distribution in the two groups using Pearson chi-square.

We used logistic regression analysis to examine the relationship between reflective ability and professionalism lapses. The dependent variable was whether or not a student had been cited for a professionalism deficiency during medical school. The independent variables were the student’s (1) gender, (2) age at matriculation, and (3) reflection score. All independent variables were entered into the model in one step.

The Indiana University institutional review board deemed the research protocol exempt from full board review.

Back to Top | Article Outline

Results

Ninety-two students from the study population had been cited for competency deficiencies related to professionalism; however, 22 of these students did not have a professionalism journal entry for us to score and thus were excluded from the study. The remaining 70 students composed the case group. The control group included 230 randomly selected students who had no SPC history but at least one journal entry for us to score. Characteristics of the two groups are summarized in Table 3. The average age at matriculation and gender distribution of the control group were not significantly different from those of the overall student population at IUSM during the study period.14

Table 3

Table 3

Results of the t tests are presented in Table 3. Students in the case group had significantly lower reflection scores than students in the control group (case group = 2.46 ± 1.05 versus control group = 2.82 ± 0.83; P = .01).

Logistic regression analysis revealed that reflection score was significantly associated with professionalism lapses among the medical students in our sample. In the regression model, reflection score was considered a continuous variable, with values ranging from 0 to 6; therefore, the odds ratio indicates the change in the likelihood that a student had been cited for a professionalism lapse that would result from a one-point increase in reflection score. An odds ratio of less than 1 indicates a decreased likelihood, while an odds ratio of greater than 1 indicates an increased likelihood. The odds ratio for reflection score was 0.64 (95% confidence interval [CI]: 0.47–0.87; P < .01; see Table 4), which indicates that as a student’s reflection score increased, the odds that she or he had been cited for a professionalism lapse decreased. Consequently, students who had lower reflection scores were more likely to have been cited for professionalism lapses. This increase in likelihood of having been cited for a professionalism lapse can be expressed by the inverse of the odds ratio (1/0.64 = 1.56; 95% CI: 1.15–2.13). Thus, for every 1-point decrease in reflection score, a student was 1.5 times more likely to have been cited for a professionalism lapse during medical school. Neither gender nor age at matriculation was associated with likelihood of having been cited for a professionalism lapse.

Table 4

Table 4

Back to Top | Article Outline

Discussion

Current literature supports that reflective ability is an essential component of professionalism.2–4 , 8 A lack of reflection among practitioners has been cited as a potential cause for some lapses in professionalism.29 Our study provides empirical evidence to support these claims by showing that lower reflection scores are associated with an increased likelihood of having been cited for a professionalism lapse during medical school. Although the difference in reflection scores between the two groups in our study was small, the higher scores in the control group suggest that those students considered the justification or evidence of their learning (based on the rubric’s criteria), even if that evidence was based solely on their own personal opinion. The slightly lower scores in the case group suggest that those students accepted at face value what they were taught, without considering the evidence supporting it. We also should point out that the overall reflection scores in our study are consistent with those from previous studies using this rubric.22–25

Although the findings of this study point to an association between reflection and professionalism, the exact nature of this relationship is still unclear. A survey of IM clerkship directors revealed that most believed that reflection enhances students’ professionalism, and thus they used reflective writing as a learning activity during their clerkship.12 This notion is also the foundation of narrative medicine, in which physicians seek to understand patients’ stories of illness to enhance their empathy, professionalism, and trustworthiness.30 After participating in a narrative medicine elective, students perceived that engaging in reflection had helped their development both personally and professionally by increasing their capacity for empathy and for providing patient-centered care.31 Such anecdotal claims aside, little empirical evidence supports the use of reflection as a tool to enhance professionalism, so further research is needed in this area.32 , 33

Others view reflection as a means for students to demonstrate their competence by providing evidence of their personal and professional growth and learning.34 Prior studies of reflective writing largely have relied on qualitative methods to gain insights into students’ professional attitudes and values35 , 36; however, these methods can be time consuming and subjective in nature. A quantitative approach could provide a more efficient and reliable method to assess professionalism; however, it first would require substantial evidence that reflective ability is a valid proxy measure for professionalism. This study provides some evidence to support this perspective, as students with higher reflection scores were also found to be “more professional” (as evidenced by their lack of citations for professionalism lapses). These findings are far from conclusive, but they provide a foundation for further research on this topic.

Back to Top | Article Outline

Limitations

This study has several limitations. First, it was conducted at a single institution; therefore, these findings may not be generalizable to other medical schools. Second, the professionalism journals captured students’ reflective ability at only one point in time; thus, it is unclear whether these journals are an accurate representation of the students’ reflective abilities as we did not evaluate other writing samples to assess their reliability.

We also acknowledge that including self-awareness and moral reasoning as professionalism domains may have inflated the relationship between reflection and professionalism because these traits may be considered inherent in one’s ability to reflect. However, our analysis of those students who were cited only for professionalism lapses (n = 57) yielded similar results.

Next, this study was retrospective and relied only on data that were available at the time of data collection; for example, 22 students did not have a professionalism journal entry to score and therefore were not included in our analysis. Inquiries to the student affairs office revealed that 8 of these students had been dismissed or withdrew from IUSM before completing their IM clerkship. We could not find an explanation for the remaining 14 students. Perhaps their failure to submit a professionalism journal entry reflects a cavalier attitude toward personal accountability, which contributed to their citation in the first place. We selected the control group from only those students who had submitted journal entries, so we do not know how many potential control group students did not submit journal entries or if this proportion differs from that of the case group.

The timing of the professionalism deficiency citations and remediation activities is another confounding factor, because students who are cited for professionalism lapses are often required to engage in reflective writing as part of their remediation.37 If the citation occurred before the third-year IM clerkship, this exposure to reflective writing may have had some influence on students’ reflective ability, which in turn affected their professionalism journal entries. As students in the case group still had significantly lower reflection scores, this factor likely did not affect the results of our study. Finally, we acknowledge that any number of other confounding factors, such as students’ background or personality characteristics, might have had some influence on their reflective ability and/or professionalism.

Back to Top | Article Outline

Conclusions

We found a significant association between medical students’ reflective ability and their professionalism, although we cannot infer a causal relationship from our findings. Nonetheless, reflection appears to be a valuable addition to the medical curriculum, as a means of helping students develop professionalism and as a means by which students can demonstrate their professionalism. Another interpretation of our findings is that this is not a linear, causal relationship but, rather, a circular relationship in which one factor enhances the other. Regardless of their purpose, reflective activities must be effectively integrated into the curriculum, lest they be perceived as disconnected from the overall educational experience. For students to recognize the importance of reflection, activities must be accompanied by a culture that values reflection and aims to develop reflective practitioners.8

Acknowledgments: The authors would like to thank Dr. Gary Pike for his assistance with the statistical analysis.

Back to Top | Article Outline

References

1. American Board of Internal Medicine. Project Professionalism. 1995.Philadelphia, Pa: American Board of Internal Medicine.
2. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226–235.
3. Swick HM. Toward a normative definition of medical professionalism. Acad Med. 2000;75:612–616.
4. Leach DC. Transcendent professionalism: Keeping promises and living the questions. Acad Med. 2014;89:699–701.
5. Schön DA. The Reflective Practitioner: How Professionals Think in Action. 1983.New York, NY: Basic Books.
6. Park J, Woodrow SI, Reznick RK, Beales J, MacRae HM. Observation, reflection, and reinforcement: Surgery faculty members’ and residents’ perceptions of how they learned professionalism. Acad Med. 2010;85:134–139.
7. Aukes LC, Geertsma J, Cohen-Schotanus J, Zwierstra RP, Slaets JP. The development of a scale to measure personal reflection in medical practice and education. Med Teach. 2007;29:177–182.
8. Sandars J. The use of reflection in medical education: AMEE guide no. 44. Med Teach. 2009;31:685–695.
9. Mezirow J. A critical theory of adult learning and education. Adult Educ Q. 1981;32:3–24.
10. Fischer MA, Harrell HE, Haley HL, et al. Between two worlds: A multi-institutional qualitative analysis of students’ reflections on joining the medical profession. J Gen Intern Med. 2008;23:958–963.
11. Ginsburg S, Regehr G, Stern D, Lingard L. The anatomy of the professional lapse: Bridging the gap between traditional frameworks and students’ perceptions. Acad Med. 2002;77:516–522.
12. Chretien KC, Chheda SG, Torre D, Papp KK. Reflective writing in the internal medicine clerkship: A national survey of clerkship directors in internal medicine. Teach Learn Med. 2012;24:42–48.
13. Litzelman DK, Cottingham AH. The new formal competency-based curriculum and informal curriculum at Indiana University School of Medicine: Overview and five-year analysis. Acad Med. 2007;82:410–4l21.
14. Brokaw JJ, Torbeck LJ, Bell MA, Deal DW. Impact of a competency-based curriculum on medical student advancement: A ten-year analysis. Teach Learn Med. 2011;23:207–214.
15. Hilton SR, Slotnick HB. Proto-professionalism: How professionalisation occurs across the continuum of medical education. Med Educ. 2005;39:58–65.
16. Lavin B, Pangaro L. Internship ratings as a validity outcome measure for an evaluation system to identify inadequate clerkship performance. Acad Med. 1998;73:998–1002.
17. Roop SA, Pangaro L. Effect of clinical teaching on student performance during a medicine clerkship. Am J Med. 2001;110:205–209.
18. Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav Res Methods. 2009;41:1149–1160.
19. O’Sullivan P, Aronson L, Chittenden E, Niehaus B, Learman L. Reflective ability rubric and user guide. MedEdPORTAL. 2010. https://www.mededportal.org/publication/8133. Accessed December 21, 2015.
20. Boud D, Keogh R, Walker D. Reflection: Turning Experience Into Learning. 1985.New York, NY: Nichols Publishing Company.
21. Mezirow J. On critical reflection. Adult Educ Q. 1998;48:185–198.
22. Aronson L, Niehaus B, DeVries CD, Siegel JR, O’Sullivan PS. Do writing and storytelling skill influence assessment of reflective ability in medical students’ written reflections? Acad Med. 2010;85(10 suppl):S29–S32.
23. Aronson L, Niehaus B, Hill-Sakurai L, Lai C, O’Sullivan PS. A comparison of two methods of teaching reflective ability in year 3 medical students. Med Educ. 2012;46:807–814.
24. Aronson L, Niehaus B, Lindow J, Robertson PA, O’Sullivan PS. Development and pilot testing of a reflective learning guide for medical education. Med Teach. 2011;33:e515–e521.
25. Learman LA, Autry AM, O’Sullivan P. Reliability and validity of reflection exercises for obstetrics and gynecology residents. Am J Obstet Gynecol. 2008;198:461.e1–461.e10.
26. Learman LA. Chair of obstetrics and gynecology, Indiana University School of Medicine. E-mail communication with L.A. Hoffman, August 2012.
27. Cohen J. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull. 1968;70:213–220.
28. Stratton TD, Elam CL. A holistic review of the medical school admission process: Examining correlates of academic underperformance. Med Educ Online. 2014;19:22919.
29. Epstein RM. Mindful practice. JAMA. 1999;282:833–839.
30. Charon R. The patient–physician relationship. Narrative medicine: A model for empathy, reflection, profession, and trust. JAMA. 2001;286:1897–1902.
31. Arntfield SL, Slesar K, Dickson J, Charon R. Narrative medicine as a means of training medical students toward residency competencies. Patient Educ Couns. 2013;91:280–286.
32. Baernstein A, Fryer-Edwards K. Promoting reflection on professionalism: A comparison trial of educational interventions for medical students. Acad Med. 2003;78:742–747.
33. Stark P, Roberts C, Newble D, Bax N. Discovering professionalism through guided reflection. Med Teach. 2006;28:e25–e31.
34. Friedman Ben David M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJ. AMEE medical education guide no. 24: Portfolios as a method of student assessment. Med Teach. 2001;23:535–551.
35. Braun UK, Gill AC, Teal CR, Morrison LJ. The utility of reflective writing after a palliative care experience: Can we assess medical students’ professionalism? J Palliat Med. 2013;16:1342–1349.
36. Howe A, Barrett A, Leinster S. How medical students demonstrate their professionalism when reflecting on experience. Med Educ. 2009;43:942–951.
37. Frankel RM. Feldman MD, Christensen JF. Professionalism.Behavioral Medicine: A Guide for Clinical Practice. 2008:3rd ed. New York, NY: McGraw-Hill; 424–430.
© 2016 by the Association of American Medical Colleges