As key stakeholders in the learning environment, adult learners are uniquely positioned to provide input on continuous quality improvement (CQI) initiatives and educational reforms. Unfortunately, medical schools have generally underused students as educational consultants and may inadequately engage them in the curricular design process.1
A number of medical schools have used curricular reform as an opportunity to formalize student involvement in medical education. Most of these schools, however, offer a descriptive account of how students are involved with their curricular changes without a full assessment of the program’s acceptability or sustainability.2–5 These descriptive pieces nonetheless offer valuable blueprints for engaging medical students in continuous curricular improvement and informed our local innovation. We build on this work by systematically assessing program use and acceptability among key stakeholders, as these factors are critical to program sustainability and generalizability.
In fall 2015, Harvard Medical School (HMS) implemented Pathways, a new curriculum for undergraduate medical education.6 This implementation involved a complete transformation of curricular structure, pedagogy, content, and teaching roles, including the (1) implementation of a case-based collaborative learning (CBCL) preclerkship curriculum delivered by core teaching faculty, (2) integration of earlier clinical exposure, and (3) movement of the clerkship year from the third to the second year. Additionally, this transformation included changes to student assessment and the development of postclerkship integrated science courses. The Pathways Education Representatives (Ed Reps) program was created simultaneously to address the inevitable challenges that come with the deployment of a new curriculum and to accelerate CQI by engaging the new curriculum’s end users—students—in this process.
To develop the Ed Reps program in anticipation of the Pathways curriculum launch, HMS faculty examined existing external and internal student engagement initiatives. This included a review of the previously cited experiences at other medical schools,2–5 as well as meeting with the Harvard Business School’s Educational Representatives.7 At HMS, students historically provided feedback on the curriculum through participation on official governance committees and completion of end-of-course evaluations (Table 1). In 2014, students who participated in the pilot for the new CBCL preclerkship curriculum8 used human-centered design principles to devise a system for providing in-person consultation to faculty developing the curriculum (the Curriculum Consultants pilot).9 Some of these students, along with student council leadership, worked together to create and pilot an online anonymous system for capturing real-time feedback.
The Curriculum Consultants pilot demonstrated the value of student–faculty partnerships to address curricular challenges and the benefits of in-person consultations, and is now run as a program on an as-needed basis (Table 1). Building on these lessons, the Ed Reps program was created in 2015 to launch alongside the new curriculum with the aim of fostering a culture that promoted a greater sense of partnership between faculty and students for continuous and real-time curricular improvement. The Ed Reps program differed from previous internal efforts by placing students at the center of the CQI process and institutionalizing a communication mechanism for bidirectional feedback between faculty and trained student representatives in a timely, efficient manner. This real-time student–faculty partnership centered around CQI represented a major shift from the existing complement of programs (see above) and aimed to support the frequent refinement of course content and pedagogy that was anticipated to be needed following the launch of a new curriculum.
Ed Reps selection, faculty mentorship, and training
Students are selected into this voluntary, application-based program within two months of matriculation with the expectation that they will continue their service longitudinally through their clerkship year and beyond. All students who applied to the Ed Reps program in the first two years were selected (21 in academic year 2015–2016 [AY15] and 19 in academic year 2016–2017 [AY16]). Starting in the third year (academic year 2017–2018 [AY17]), fewer slots were offered (8 in total), and senior Ed Reps ranked candidates based on their applications.
A faculty mentor provides support by participating in the Ed Reps’ biweekly meetings (see below) and is available to meet with Ed Reps on an ad hoc basis. The mentor also provides training in adult learning theory, the evidence-based teaching strategies underpinning the Pathways curriculum, best practices for providing feedback to faculty and peers, and procedures for connecting fellow students to institutional supports as needed (e.g., the disability office, counseling and mental health services, anonymous learning environment reporting tools). These trainings were provided informally in AY15 and were developed into structured modules for AY16 and beyond.
To streamline communication with course directors, one or two Ed Reps served as the lead contact for each preclerkship course and for each of four learning communities. A learning community comprises 40 students and two lead faculty. During each CBCL session in the preclerkship course, students work collaboratively in groups of four to apply new knowledge to in-class cases, after which faculty lead a review of the cases with the entire 40-person learning community. Each learning community uses the same preparatory work, cases, in-class materials, and assessments for each session to ensure that the curriculum is implemented consistently.
Ed Reps sought real-time student feedback about course content, pedagogy, classroom dynamics, assessments, and their impact on learning. Although methods for gathering feedback evolved, feedback was typically gathered through optional biweekly anonymous online surveys in addition to informal peer conversations and learning community meetings. The Ed Reps met biweekly to review the gathered feedback, identify challenges, and generate solutions; met regularly with course directors and core faculty to convey bidirectional feedback (see below) to optimize the learning environment in real time; and updated their class on faculty responses to student feedback via weekly emails and in-person announcements. Ed Reps also periodically reviewed faculty feedback on students’ level of preparation for and engagement in the learning environment, suggestions for effective study habits, and the scientific rationale behind the Pathways curriculum with their peers. Additional roles were created as the Ed Reps program evolved to meet CQI needs, including working groups of Ed Reps dedicated to broader initiatives, such as the attendance policy and enhancing diversity in the curriculum.
We present data from the Ed Reps impact survey project, which includes three student surveys and one faculty survey. Administered in the final month of the preclerkship year, the student survey assessed awareness of, use of, and satisfaction with the Ed Reps program and asked for free-text comments on program strengths and weaknesses for the inaugural three years of the program (AY15–AY17). All students in the Pathways curriculum, excluding Ed Reps, were invited to respond. Core faculty who taught in the preclerkship curriculum in the first two years of the program (AY15 and AY16) were surveyed about the program at the conclusion of AY16. The faculty survey included questions on the degree of interaction with Ed Reps and adjustments to courses based on Ed Reps’ feedback, and asked for free-text comments on overall impressions of the program.
Respondents to all surveys chose from a five-point scale (from “strongly agree” to “strongly disagree”) to indicate their level of agreement with statements regarding the Ed Reps program. All data presented summarize the percentage of respondents in agreement with a statement (i.e., who answered either “somewhat agree” or “strongly agree”). We performed chi-square tests to detect significant statistical differences (P < .05) by year. All surveys were optional and anonymous. The Academy at HMS and Office of Human Research Administration reviewed this study and deemed that it met criteria for institutional review board exemption.
A total of 222 Pathways students completed the survey across the initial three years of the program (AY15: 77/152 [50.7% response rate (RR)]; AY16: 68/145 [46.9% RR]; AY17: 77/159 [48.4% RR]; P = .94). As there were no statistically significant differences in responses across academic years, we provide the combined quantitative student results below and the results stratified by year in Supplemental Digital Appendix 1 (at http://links.lww.com/ACADMED/A661). The vast majority of students were aware of their Ed Reps (217; 97.7%), and most students reported providing feedback to their Ed Reps at some point in the year (156; 70.3%). A strong majority of respondents reported that Ed Reps had a positive impact on the curriculum (202; 91.0%), facilitated student feedback to faculty (212; 95.5%), and helped make real-time improvements to ongoing courses (194; 87.4%). Relatively few students felt that the Ed Reps were a barrier to direct communication with faculty (24; 10.8%).
Several central themes emerged from the students’ free-text responses (Table 2). Students appreciated that Ed Reps facilitated timely, synthesized feedback to faculty by offering a structured path for student input. Student empowerment was valued, with students noting the real-time manner in which course changes were implemented based on their input. Some students suggested a need to increase the transparency of the Ed Reps’ feedback solicitation and communication processes to ensure that feedback was more representative of the entire class.
A total of 47 (out of 64 eligible) faculty members completed the survey (73.4% RR), most of whom taught in the Pathways curriculum in both AY15 and AY16 (42/47; 89.4%). Of faculty who reported interacting with Ed Reps in one or both years, the majority agreed that the Ed Reps had a positive impact on the preclerkship curriculum (AY15: 26/32 [81.3%]; AY16: 32/35 [91.4%]; P = .22). Additionally, a majority reported that the Ed Reps facilitated constructive student feedback to faculty (AY15: 28/32 [87.5%]; AY16: 31/35 [88.6%]; P = .89) and faculty feedback to students (AY15: 21/31 [67.7%]; AY16: 22/33 [66.7%]; P =.93), enabled real-time improvements to ongoing courses (AY15: 27/32 [84.4%]; AY16: 29/35 [82.9%]; P = .87), and helped facilitate changes for future iterations of the curriculum (AY15: 25/32 [78.1%]; AY16: 29/35 [82.9%]; P = .63; see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A661).
The majority of faculty agreed that the Ed Reps had a positive impact in the following areas: the learning environment (34/37; 91.9%), student–faculty relationships (33/37; 89.2%), and teaching and learning activities (32/36; 88.9%). Although still a majority, fewer faculty agreed that the Ed Reps had a positive impact on learning objectives (19/33; 57.6%) and institutional rules and procedures (16/30; 53.3%). Of faculty who received feedback from the Ed Reps, almost all reported making changes to their course based on this input (35/37; 94.6%). Specific domains of curricular change reported by faculty for ongoing and future courses are shown in Figure 1.
Data from faculty’s free-text responses highlighted central themes regarding strengths of the program, including appreciation for student–faculty collaboration, student empowerment, and the timeliness of feedback, which increased opportunity for real-time curricular improvement. Areas for improvement included ensuring that feedback was representative of the broader class, clarifying how Ed Reps solicit feedback, and refining their role in supporting curricular learning objectives (Table 2).
Program awareness, use, and satisfaction
Awareness of, use of, and satisfaction with the Ed Reps program was high among faculty and students. A significant majority of students felt that the Ed Reps had a positive effect on the curriculum, facilitated student feedback to faculty, and helped make real-time improvements to ongoing courses. Importantly, students’ perspectives were echoed in faculty responses, with the majority of faculty reporting positive responses to the program. Faculty viewed the impact of Ed Reps in promoting a positive learning environment and facilitating student–faculty partnerships favorably, suggesting that the program helped to achieve the intended culture change: a true sense of partnership in creating the new curriculum. Our results demonstrate that the Ed Reps program also achieved its objective of facilitating real-time curricular improvement, with nearly 95% of faculty reporting making curricular changes in response to Ed Reps’ feedback.
Limitations of our evaluation include its retrospective nature and the modest response rate among students, the latter of which is likely attributable to survey fatigue. Consequently, our results may be susceptible to response bias, though we would have expected lower satisfaction had only individuals with strong negative opinions been drawn to respond. Both the student and faculty surveys are subject to recall bias, potentially leading to an overrepresentation of specific salient interactions with the Ed Reps rather than representing the totality of their experiences with Ed Reps. Further, our study design lacks a true comparison group. However, because the first year of the program represented the initial curricular rollout, and years two and three represent the start of the maintenance phase of the new curriculum, comparing results among these groups demonstrates program acceptability at different phases of curricular reform.
Challenges and future directions
As we continue to distribute the Ed Reps impact survey to students annually, the year-to-year comparisons will be used for CQI of the Ed Reps program itself (i.e., to identify, track, and address any concerning trends). One example of CQI resulting from the impact survey is the more rigorous Ed Reps selection process that requires candidates to share their approach to five scenarios they might encounter as an Ed Rep, which was implemented in AY17. Additionally, Ed Reps continue to use these impact survey data to improve their methods for collecting peer feedback and conveying faculty responses back to their peers (e.g., via standardized communications processes such as structured weekly email updates). Further, the Ed Reps program was recently integrated into the HMS Office of Educational Quality Improvement, thereby institutionalizing the role of the program for future classes, providing administrative support, and facilitating opportunities to assess other curricular outcomes. Future work will include optimally leveraging the process for CQI in the clerkship and postclerkship phases and increasing mentorship from senior to novice Ed Reps. The high rates of program satisfaction, awareness, and use demonstrated here should encourage other schools to employ CQI when implementing new strategies for student curricular engagement.
Acknowledgments: The authors wish to acknowledge the invaluable contributions of all former and current members of the Education Representatives (Ed Reps) program who have been part of shaping this initiative. Further, they wish to thank all faculty and peers who have provided input on how the program can better serve the goal of improving medical education.
1. Burk-Rafel J, Jones RL, Farlow JL. Engaging learners to advance medical education. Acad Med. 2017;92:437–440.
2. Fetterman DM, Deitz J, Gesundheit N. Empowerment evaluation: A collaborative approach to evaluating and transforming a medical school curriculum. Acad Med. 2010;85:813–820.
3. Goldfarb S, Morrison G. Continuous curricular feedback: A formative evaluation approach to curricular improvement. Acad Med. 2014;89:264–269.
4. Hsih KW, Iscoe MS, Lupton JR, et al. The student curriculum review team: How we catalyze curricular changes through a student-centered approach. Med Teach. 2015;37:1008–1012.
5. Yengo-Kahn AM, Baker CE, Lomis AK. Medical students’ perspectives on implementing curriculum change at one institution. Acad Med. 2017;92:455–461.
6. Harvard Medical School. Pathways. https://meded.hms.harvard.edu/pathways
. Accessed March 6, 2019.
8. Krupat E, Richards JB, Sullivan AM, Fleenor TJ Jr, Schwartzstein RM. Assessing the effectiveness of case-based collaborative learning via randomized controlled trial. Acad Med. 2016;91:723–729.
9. Anderson J, Calahan CF, Gooding H. Applying design thinking to curriculum reform. Acad Med. 2017;92:427.