Gauging Events That Influence Students’ Perceptions of the Medical School Learning Environment: Findings From One Institution
Shochet, Robert B. MD; Colbert-Getz, Jorie M. PhD; Levine, Rachel B. MD, MPH; Wright, Scott M. MD
Dr. Shochet is assistant professor, Department of Medicine, and director, Colleges Advisory Program, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Dr. Colbert-Getz is director, Office of Assessment and Evaluation, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Dr. Levine is associate professor, Department of Medicine, and core faculty, Colleges Advisory Program, Johns Hopkins University School of Medicine, Baltimore, Maryland.
Dr. Wright is professor of medicine and director, Division of General Internal Medicine, Johns Hopkins Bayview Medical Center, Johns Hopkins University School of Medicine, Baltimore, Maryland.
First published online December 23, 2012
Correspondence should be addressed to Dr. Shochet, Johns Hopkins University School of Medicine, Anne and Mike Armstrong Medical Education Building, 1600 McElderry St., Suite 202, Baltimore, MD 21205; telephone: (410) 502-3737; fax: (410) 502-0167; e-mail: email@example.com.
Purpose: The medical school learning environment (LE), encompassing the physical, social, and psychological context for learning, holds significant influence on students’ professional development. Among these myriad experiences, the authors sought to gauge what students judge as influencing their perceptions of the LE.
Method: Fourth-year medical students at Johns Hopkins University participated in this cohort survey study before their 2010 graduation. A list of 55 events was iteratively revised and pilot-tested before being administered online. Responses assessed whether students experienced each event and, if so, the degree of impact on perceptions of the LE. A calculated mean impact score (MIS) provided a means to compare the relative impact of events.
Results: Of 119 students, 84 (71%) completed the survey. Students rated the overall LE as exceptional (29/84; 35%), good (36/84; 43%), fair (17/84; 20%), or poor (2/84; 2%). Eighty percent of students experienced at least 41 of the 55 events. MIS values ranged from 2.00 to 3.76 (highest possible: 4.00). Students rated positive events as having the highest impact. Few significant differences were found across gender, age, or surgical/nonsurgical specialty choice. MIS distributions differed between those perceiving the LE as exceptional or fair to poor for 22 (40%) of 55 events.
Conclusions: This study attempted to identify the discrete events that medical students perceive as most affecting their sense of the LE. Knowing the phenomena that most strongly influence student perceptions can inform how settings, relationships, and interactions can be shaped for meaningful learning and professional formation.
The learning environment (LE) in medical schools and at academic health centers holds significant influence on medical students’ professional development, affecting how students form their identities and moral constructs and pattern their behaviors.1–3 The LE encompasses the physical, social, and psychological context in which students learn; all interactions with faculty, staff, and peers; and the formal, informal, and hidden curricula.4,5 Supportive medical school LEs are associated with enhanced student learning, achievement, and humanism,6,7 although undercurrent messages conveyed through the hidden curriculum may erode the values and behaviors taught in more formal settings.8,9 In a recently acclaimed text on the future of medical education, Cooke and colleagues10 advocate for a more intentional shaping of the medical school LE as a means to enhance students’ professional formation, creating a greater sense of coherence between espoused and enacted values.
The Liaison Committee on Medical Education highlights the importance of the LE in its accreditation standard MS-31A11:
A medical education program must ensure that its learning environment promotes the development of explicit and appropriate professional attributes in its medical students … [and should] regularly evaluate the learning environment to identify positive and negative influences on the maintenance of professional standards and conduct and develop appropriate strategies to enhance the positive and mitigate the negative influences.
Prior studies identified that medical students’ perceptions of their LE are shaped by discrete stressors,12 as well as discrete events they experience, that then mediate their humanistic practices,13 sense of well-being,14 or professional burnout.15 In a recent study by Murinson and colleagues,16 evocative events experienced by medical students were studied to assess how they might influence students’ emotional development. We hypothesized that students’ perceptions of the LE are mediated by the impact of the discrete events they experience in medical school. This study attempts to characterize the events that students believe are most influential in determining how they perceive and are influenced by their LE.
We invited fourth-year medical students from the Johns Hopkins University School of Medicine (JHUSOM) Class of 2010 to participate in this cohort survey study.
Instrument development and data collection
The study questionnaire was developed and iteratively revised over a period of several months. Drawing from the work of our study team14 who identified 70 discrete events or relationships that medical students might experience, we assembled and iteratively revised a similar list of potential student experiences. Our team has more than 50 combined years working closely with medical students, which includes 7 years coordinating the school’s student learning community and advising program. We further refined the event list by literature review, input from the medical school’s learning community’s core faculty, and discussion at our medical education research-in-progress conference. Ultimately, the research team reached consensus on an event list containing 55 items. We devised a response option scale to capture two features simultaneously: whether or not a student experienced the event or relationship during medical school, and, if experienced, the degree to which it affected the student’s perception of the LE. Five response options were provided for each item: “No, not experienced,” “Yes, experienced, but it had no impact,” “Yes … and it had low impact,” “Yes … and it had moderate impact,” and “Yes … and it had a high impact.” The internal consistency of the items was measured with Cronbach alpha.
We incorporated the 55-item event list and LE definition into the annual online survey administered to students by the JHUSOM Colleges Advisory Program. This survey also included questions about advising experiences, professional growth, and career choice in medical school. The online survey link was sent to fourth-year students (henceforth termed “graduating students”) via e-mail in the first week of May 2010, three weeks before their graduation. Students participated on a voluntary basis, and those completing the survey were entered into a drawing to win one of five restaurant gift cards. The study was approved by a JHUSOM institutional review board.
We analyzed the data using IBM SPSS software version 19 (SPSS Inc., Armonk, New York). Respondent characteristics are presented as proportions and means. To evaluate the impact of each of the 55 discrete events on students’ perceptions of the LE, we measured both event prevalence and mean impact. We calculated prevalence as the percentage of students choosing impact ratings other than “not experienced.” We calculated impact by assigning numerical values to the verbal descriptors as follows: 1 = experienced, no impact; 2 = experienced, low impact; 3 = experienced, moderate impact; and 4 = experienced, high impact. The numerical values associated with the students’ responses were then summed for each event and divided by the total number of students who experienced the event. We defined this result as the mean impact score (MIS). To analyze the distributions of responses for those perceiving the overall LE as exceptional compared with those who viewed it as fair or poor, as well as by demographic factors (gender, age, and surgical or nonsurgical specialty choice), we used the Mann–Whitney U test. Finally, we calculated the difference in MIS values, or “delta impact,” between the “exceptional” and “fair or poor” subgroups.
Characteristics of the study population and overall assessment of the LE
Of 119 graduating students, 92 (77%) responded to the questionnaire. Of these, we analyzed data for the 84 students (71%) who responded to all 55 events on this subsection of the survey. Demographic characteristics of the study population are shown in Table 1. Students completing the survey in full (N = 84) did not differ significantly by age (P = .965), gender (P = .708), or race (P = .409) compared with nonresponders and those completing only a portion of the survey (N = 35). Students rated their overall perception of the LE as follows: exceptional (29/84; 35%), good (36/84; 43%), fair (17/84; 20%), poor (2/84; 2%), or terrible (0/84; 0%).
Prevalence and impact scores of events
Table 2 shows the 55 medical school events, experiences, and relationships presented to graduating students for their reporting of prevalence of experiencing the event, calculated MIS with standard deviation, and fraction of students reporting each event as having high impact. The prevalence of experiencing these 55 events ranged from 18% to 100%, with an overall mean of 87%. More than 50% of the 84 respondents experienced 54 of the 55 events, and more than 80% of respondents experienced 41 events. The MIS of the 55 events ranged from 2.00 to 3.76 (out of a possible highest score of 4.00) with an overall mean of 3.00. MIS scores for the 55 events had a Cronbach alpha of 0.86.
When examining differences in distributions of student responses to the 55 events on the basis of surgical or nonsurgical specialty choice, gender, or age, only a few statistically significant differences were noted. Among graduating students declaring surgical versus nonsurgical specialty, only one event achieved statistical significance: “participating in a research project with JHUSOM faculty” (nonsurgical MIS = 3.71 versus surgical MIS = 3.38, U = 542, P = .023). Three significant differences were related to student gender: “encountering negative role models” (female MIS = 3.00 versus male MIS = 2.54, U = 580.5, P = .030), “working with enthusiastic and motivating teachers” (female MIS = 3.84 versus male MIS = 3.55, U = 685, P = .037), and “witnessing a patient being treated disrespectfully” (female MIS = 3.29 versus male MIS = 2.84, U = 481.5, P = .044). When comparing younger and older students (younger than age 26 versus older than age 26), no significant differences in response distributions were noted.
Perception of event impact as function of overall LE rating
Table 3 displays how distributions in responses differed between 29 students rating the overall LE as exceptional compared with 19 students rating it as fair or poor. Twenty-two of the 55 events (40%) were significantly different in distributions of responses between these subgroups. Of these 22 events, 18 were positively charged and received significantly higher MIS ratings by students perceiving the LE as exceptional compared with those perceiving it as fair or poor. Conversely, 4 negatively charged events received significantly higher MIS ratings by students rating the overall LE as fair or poor compared with those perceiving it as exceptional.
Social learning theory, as originally described by Bandura,17 posits that individuals learn within a social context, through continuous reciprocal interactions between cognitive, behavioral, and environmental determinants. This study describes how a range of events and relationships experienced by most students in medical school differentially affect their perceptions of the LE. Graduating JHUSOM students affirmed both high prevalence and impact for most of the 55 events presented. Of these, the 15 events receiving the highest MIS scores by students can be characterized as positive, appreciative experiences or relationships involving clinical teams, faculty, patients, and peers.
Characterizing and harnessing the complex forces within the medical school LE to bring a greater sense of coherence and humanism to the training of physicians has been the subject of numerous studies and reports.4–7,18–23 The Dundee Ready Education Environment Measure, a 50-item student survey, has been used to “profile” an institution’s strengths and weaknesses across domains of learning, teaching, atmosphere, and student self-perceptions, to compare global scores across institutions, and to correlate student achievement to LE perception.19 Other LE surveys have focused on teacher–learner and patient–physician relationships,20 student mistreatment,21 professionalism,22 patient-centeredness,23 and humanism.13 Not only do perceptions of the LE vary across institutions24 but students at a single institution also may respond differently to the experience of similar events.25
The majority of students in our study judged the overall LE as positive, and the observed pattern of responses suggests that positive, relational experiences with teachers, patients, and peers carried a greater weight in how they perceived the LE than did negative experiences. This concept of appreciative engagement has been associated with students’ satisfaction with their educational experience.2 A learner’s ability to tap into the plentiful resources available in a medical school LE such that she or he becomes engaged, forms relational connections, and perceives a sense of belonging may be necessary to view the LE as healthy.5,26 Suchman27 asserts that when a sense of belonging and connection occur, it leads a learner to feel part of a larger whole, such as a clinical team. Students affirmed the potent impact of positive interactions with faculty on their perceptions of the LE. Three-quarters of respondents rated encountering inspiring role models as highly influential, consistent with prior studies in this regard.16,28,29 Experiences with enthusiastic teachers, dedicated mentors, and trustful advisors were also rated as highly influential. Teacher–learner relationships that extend beyond the classroom provide opportunities for deeper connections, enable “undiscussable issues” to be discussed, and allow for stress to be mitigated.30–32 Encountering institutional culture, though experienced by all students in this study, was perceived as highly influential by just 37%, suggesting that omnipresent messages connected to the hidden curriculum8 may be not be strongly influencing perceptions of the LE for the majority of students.
As a means to better support students’ professional growth and create a sense of wholeness to student life,33 learning communities have been created at many U.S. medical schools over the past decade.34,35 At its core, a learning community enhances longitudinal relationships between students and faculty, with programs focusing on clinical skills teaching, advising, community service, and well-being.36,37 A learning community can aid students to feel more connected to their LE, institution, peers, and faculty.34 Relational continuity with faculty offers the opportunity for iterative dialogue and reflection, potentially mitigating the impact of the complex events that students encounter in their training.38–40 Additionally, knowing the events that most powerfully affect student perceptions can assist institutional leaders, faculty, and resident teachers in enhancing the value students can cultivate from the LE.
Although few gender differences were noted, certain perceptions of the LE may be influenced by gender. Several recent studies demonstrate that the experiences of female medical students and trainees are shaped by gender expectations, which influence relationships, behaviors, self-confidence, self-assessment, and career planning.41–44 How female students navigate the LE may thus be different from how males do, and parallel findings have been described for female faculty in academic health centers.45 Compared with their male classmates, female students in our study reported a greater (negative) impact on LE perceptions from witnessing disrespectful patient care and encountering negative role models, whereas their perceptions were comparatively more positively influenced when encountering enthusiastic teachers. Ensuring exposure to positive role models, teachers, and mentors may help to mitigate some of the gender-based experiences that female students encounter.
We found that one-quarter of the graduating class, those rating the overall LE as fair or poor, demonstrated significantly different response patterns for 40% of the 55 events, as compared with peers rating the LE as excellent. This “fair to poor” subset was less affected by a cadre of positive events and, conversely, more strongly influenced by several negative events. This student subgroup may have heightened vulnerability to complex or difficult interactions with groups, faculty, and/or peers and potentially may be less able to build sufficient relational connectedness to meet their needs. Although associations between LE, student engagement, and academic success have been demonstrated for secondary school students,46 such associations are not as clear in medical school. Developing enhanced mechanisms to identify those students with greater likelihood of being strongly affected by negative forces within the LE would create opportunities for more individualized support.
Several limitations of the study should be considered. First, graduating students at a single medical school made up the study group, and thus our findings may have limited generalizability. Second, although the list of events included experiences occurring across the four years of medical school, administering the survey just before graduation might have resulted in recall bias, with students judging events more proximal to graduation as having greater impact than those occurring earlier in their careers. Third, data collection relied exclusively on students’ self-report. However, because perceived impact of an event is a subjective, intrapersonal construct, this method is congruent with the study’s aims. Additionally, it is possible that some respondents rated events as having a high impact for different reasons (either positive or negative). Future versions of this scale might benefit by expanding response options to include the valence of the impact on one’s perceptions. Finally, limiting the event list to 55 items may have omitted key influential events that were not identified by culling the literature, iterative review by seasoned faculty advisors, and pilot testing. Although we felt the list to be comprehensive and reliable, a list generated by students might have differed.
This study attempts to describe the events and experiences that students believe to be most influential on their perceptions of the LE and, therefore, worthy of attention. Knowing the phenomena that most strongly influence student perceptions can inform how settings, relationships, and interactions can be shaped for meaningful learning. Future studies might offer a more personalized measure of learners’ LE perceptions, thus laying a foundation for interventions to assist students in mobilizing their unique strengths and learning styles. Such measures might provide rich data for self-reflection, for faculty development in identifying student learning styles, and for coaching students in how best to adapt to their LE to enhance their professional development.
Funding/Support: This project was supported by the Johns Hopkins Osler Center for Clinical Excellence.
Other disclosures: Dr. Wright is a Miller-Coulson Family Scholar through the Johns Hopkins Center for Innovative Medicine. Dr. Shochet is an Osler Faculty Scholar.
Ethical approval: Ethical approval has been granted for studies involving human subjects by a Johns Hopkins University School of Medicine institutional review board.
1. Branch WT Jr, Pels RJ, Hafler JP. Medical students’ empathic understanding of their patients. Acad Med. 1998;73:360–362
2. Suchman AL, Williamson PR, Litzelman DK, Frankel RM, Mossbarger DL, Inui TSRelationship-Centered Care Initiative Discovery Team. . Toward an informal curriculum that teaches professionalism. Transforming the social environment of a medical school. J Gen Intern Med. 2004;19(5 pt 2):501–504
3. Branch WT Jr. Supporting the moral development of medical students. J Gen Intern Med. 2000;15:503–508
4. Genn JM. AMEE medical education guide no. 23 (part 1): Curriculum, environment, climate, quality and change in medical education—A unifying perspective. Med Teach. 2001;23:337–344
5. Hutchinson L. Educational environment. BMJ. 2003;326:810–812
6. Genn JM. AMEE medical education guide no. 23 (part 2): Curriculum, environment, climate, quality and change in medical education—A unifying perspective. Med Teach. 2001;23:445–454
7. Gracey CF, Haidet P, Branch WT, et al. Precepting humanism: Strategies for fostering the human dimensions of care in ambulatory settings. Acad Med. 2005;80:21–28
8. Hafferty FW. Beyond curriculum reform: Confronting medicine’s hidden curriculum. Acad Med. 1998;73:403–407
9. Stern DT. In search of the informal curriculum: When and where professional values are taught. Acad Med. 1998;73(10 suppl):S28–S30
10. Cooke M, Irby DM, O’Brien BC Educating Physicians: A Call for Reform of Medical School and Residency.. 2010 Stanford, Calif Jossey-Bass
12. Calkins EV, Arnold L, Willoughby TL. Medical students’ perceptions of stress: Gender and ethnic considerations. Acad Med. 1994;69(10 suppl):S22–S24
13. Moyer CA, Arnold L, Quaintance J, et al. What factors create a humanistic doctor? A nationwide survey of fourth-year medical students. Acad Med. 2010;85:1800–1807
14. Haglund ME, aan het Rot M, Cooper NS, et al. Resilience in the third year of medical school: A prospective study of the associations between stressful events occurring during clinical rotations and student well-being. Acad Med. 2009;84:258–268
15. Dyrbye LN, Thomas MR, Harper W, et al. The learning environment and medical student burnout: A multicentre study. Med Educ. 2009;43:274–282
16. Murinson BB, Klick B, Haythornthwaite JA, Shochet R, Levine RB, Wright SM. Formative experiences of emerging physicians: Gauging the impact of events that occur during medical school. Acad Med. 2010;85:1331–1337
17. Bandura A Social Learning Theory. 1977 Englewood Cliffs, NJ Prentice-Hall
18. Marshall RE. Measuring the medical school learning environment. J Med Educ. 1978;53:98–104
19. Roff S. The Dundee Ready Educational Environment Measure (DREEM)—A generic instrument for measuring students’ perceptions of undergraduate health professions curricula. Med Teach. 2005;27:322–325
20. Pololi L, Price J. Validation and use of an instrument to measure the learning environment as perceived by medical students. Teach Learn Med. 2000;12:201–207
21. Elnicki DM, Linger B, Asch E, et al. Patterns of medical student abuse during the internal medicine clerkship: Perspectives of students at 11 medical schools. Acad Med. 1999;74(10 suppl):S99–S101
22. Quaintance JL, Arnold L, Thompson GS. Development of an instrument to measure the climate of professionalism in a clinical teaching environment. Acad Med. 2008;83(10 suppl):S5–S8
23. Haidet P, Kelly PA, Chou CCommunication, Curriculum, and Culture Study Group. . Characterizing the patient-centeredness of hidden curricula in medical schools: Development and validation of a new measure. Acad Med. 2005;80:44–50
24. Haidet P, Kelly PA, Bentley S, et al.Communication, Curriculum, and Culture Study Group. Not the same everywhere. Patient-centered learning environments at nine medical schools. J Gen Intern Med. 2006;21:405–409
25. Roff S, McAleer S. What is educational climate? Med Teach. 2001;23:333–334
26. Hoffman M, Richmond J, Morrow J, Salomone K. Investigating “sense of belonging” in first-year college students. J Coll Student Retention. 2002;4:227–256
27. Suchman ALSuchman AL, Botelho RJ, Hinton-Walker P. Control and relation: Two foundational values and their consequences. In: Partnerships in Healthcare: Transforming Relational Process. 1998 Rochester, NY University of Rochester Press
28. Wright S, Wong A, Newill C. The impact of role models on medical students. J Gen Intern Med. 1997;12:53–56
29. Maudsley RF. Role models and the learning environment: Essential elements in effective medical education. Acad Med. 2001;76:432–434
30. Haidet P, Stein HF. The role of the student–teacher relationship in the formation of physicians. The hidden curriculum as process. J Gen Intern Med. 2006;21(suppl 1):S16–S20
31. Bickel J, Rosenthal SL. Difficult issues in mentoring: Recommendations on making the “undiscussable” discussable. Acad Med. 2011;86:1229–1234
32. Pololi L, Conrad P, Knight S, Carr P. A study of the relational aspects of the culture of academic medicine. Acad Med. 2009;84:106–114
33. Boyer EL Campus Life: In Search of Community. 1990 Princeton, NJ Carnegie Foundation for the Advancement of Teaching
34. Rosenbaum ME, Schwabbauer M, Kreiter C, Ferguson KJ. Medical students’ perceptions of emerging learning communities at one medical school. Acad Med. 2007;82:508–515
35. Ferguson KJ, Wolter EM, Yarbrough DB, Carline JD, Krupat E. Defining and describing medical learning communities: Results of a national survey. Acad Med. 2009;84:1549–1556
36. Stewart RW, Barker AR, Shochet RB, Wright SM. The new and improved learning community at Johns Hopkins University School of Medicine resembles that at Hogwarts School of Witchcraft and Wizardry. Med Teach. 2007;29:353–357
37. Drolet BC, Rodgers S. A comprehensive medical student wellness program—Design and implementation at Vanderbilt School of Medicine. Acad Med. 2010;85:103–110
38. Hirsh DA, Ogur B, Thibault GE, Cox M. “Continuity” as an organizing principle for clinical education reform. N Engl J Med. 2007;356:858–866
39. Chou CL, Johnston CB, Singh B, et al. A “safe space” for learning and reflection: One school’s design for continuity with a peer group across clinical clerkships. Acad Med. 2011;86:1560–1565
40. Levine RB, Shochet RB, Cayea D, Ashar BH, Stewart RW, Wright SM. Measuring medical students’ sense of community and satisfaction with a structured advising program. Int J Med Educ. 2011;2:125–132
41. Babaria P, Abedin S, Nunez-Smith M. The effect of gender on the clinical clerkship experiences of female medical students: Results from a qualitative study. Acad Med. 2009;84:859–866
42. Babaria P, Bernheim S, Nunez-Smith M. Gender and the pre-clinical experiences of female medical students: A taxonomy. Med Educ. 2011;45:249–260
43. Bartels C, Goetz S, Ward E, Carnes M. Internal medicine residents’ perceived ability to direct patient care: Impact of gender and experience. J Womens Health (Larchmt). 2008;17:1615–1621
44. Carnes M. Commentary: Deconstructing gender difference. Acad Med. 2010;85:575–577
45. Carr PL, Pololi L, Knight S, Conrad P. Collaboration in academic medicine: Reflections on gender and advancement. Acad Med. 2009;84:1447–1453
46. Fall AM, Roberts G. High school dropouts: Interactions between social context, self-perceptions, school engagement, and student dropout. J Adolesc. 2012;35:787–798
© 2013 Association of American Medical Colleges